Theo van Leeuwen On The Social Semiotics Of Time

 THE SOCIAL SEMIOTICS OF TIME

Theo van Leeuwen (leeuwen@sdu.dk)

The course will deal with time-based multimodal texts. It has two main objectives. Theoretically, it seeks to show that the semiotic resources we have for the temporal structuring of multimodal texts reflect the temporal structuring of social life. Practically, it introduces resources for analysing the temporal structuring of multimodal texts, including music and film, and for analysing the verbal and visual representation of time and timing in multimodal texts.

5 August Time and society

The lecture introduces the course and discusses its basic assumption – that the structuring of time-based multimodal texts is closely related to the way society organizes the timing of activities. It also discusses the power structures and normative discourses that regulate social timing.

12 August Rhythm (1)

Starting by discussing the differences between clock time and our body clocks, the lecture argues that the structuring of everyday social interaction and the structuring of time-based multimodal texts is primarily based on rhythm, and introduces a method for analysing the rhythmic structure of multimodal texts and the role it plays in meaning-making.

26 August Time and music (1)

This lecture discusses the relation between musical time and social time, introducing a number of key concepts, including measured and unmeasured time, regularized and non-regularized time, metronomic and non-metronomic time, and polyrhythmic and mono-rhythmic time, in each case explaining how these forms of timing create meaning, and how they can combine in complex structures.

9 September Time and film (1)

This lecture discusses the relation between filmic time and the changes in social time and timing which came about at the time film language was developed, as a result of technological innovations and new ways of understanding time (e.g. relativity theory). Key themes include the relation between past, present and future, the stretching or condensing of time and the presentation of distant events as simultaneous and connected.

23 September Time and language (1)

The lecture discusses linguistic resources for representing the timing of social activities, their relation to the way time and timing are organized in contemporary society, and the way they can be used to analyse the representation of time in discourse.

14 October Visual representations of time (1)

This lecture discusses semiotic resources for the (static) visual representation of time-based activities, showing how these resources can be used in analysing images and diagrams, and comparing the semantic reach of visual and verbal representation of time and timing.

Blogger Comments:

[1] On the model of Halliday's Systemic Functional Linguistic Theory, the dimension of time can be understood as the location and extent (duration or frequency) of the unfolding of processes. The difference between 'clock time and our body clocks' is in the processes whose rhythms provide the intervals of time measurement.

[2] Rhythm can be understood as the unfolding of a process relative to the time intervals used to measure it.

[3] The difference between musical time and social time is in the processes whose unfoldings provide the intervals of time measurement.

[4] This mistakes process for time. It is the unfolding of processes that is measured or unmeasured, regularised or non-regularised, metronomic or non-metronomic, polyrhythmic or monorhythmic. Timing is the unfolding of a process relative to temporal location and extent.

[5] From the perspective of Systemic Functional Linguistic Theory, the General Theory of Relativity models gravity as the relative contraction of space intervals and the relative expansion of time intervals with increasing proximity to a centre of mass.

[6] From the perspective of Systemic Functional Linguistic Theory, past, present and future are relative to the time of meaning-making, mental or verbal.

[7] Interpreting the General Theory of Relativity in terms of Systemic Functional Linguistic Theory, the 'simultaneity' of distant events can be understood in terms of the 'past in present' tense, because the time it takes light to travel entails that what is observed is a past event relative to the present of the observing process.

Reading An Image

Applying Kress & van Leeuwen's model of reading images to the following:

incongruously yields 

• the point of departure, christianity, as new and ideal
• the endpoint, atheism, as given and real.

Today's 'Typology Forum' Paper By Jim Martin

Paralanguage – a systemic-functional perspective

J R Martin

University of Sydney

SFL research exploring semiotic systems other than language, inspired by Kress & van Leeuwen's Reading Images (1996), has led to a surge of interest in multimodal discourse analysis. In this paper, drawing on this work, I will present some proposals for a model of paralanguage inspired by this work, and ask questions about the nature of the relationship between paralanguage and language – beginning to explore how this relationship is similar or different form that between language and other modalities of communication (asking in other words "What's 'para-' about paralanguage?"). Alongside Doran's and Johnston's presentations this opens up consideration of the nature of a functional perspective on typology which looks not just across languages but across semiotic systems – with a special focus on symbolism (i.e. formal symbolic systems such as those used in mathematics), sign language and paralanguage for this forum.

Blogger Comments:

To be clear, the model of body language that Martin and his collaborators have been using since 2009 is (their understanding of) the one I devised at the time as a Research Assistant. It was not inspired by Kress & van Leeuwen's Reading Images (1996), but by the final chapter of Halliday & Matthiessen's Construing Experience Through Meaning (1999), which relates language to other semiotic systems.

The model distinguishes three broad types of body language: protolinguistic, linguistic and epilinguistic. Protolinguistic body language is the use of the body to express the meanings of protolanguage; this does not cease with the ontogenetic move into language. Linguistic body language is the supplementary use of the body to make the meanings of language, as the raising of eyebrows in tune with the raising of tonic pitch, or the beating of hands in time with the rhythm of speech. Epilinguistic body language is the (non-linguistic) use of the body to express the meanings made possible by language, the analogue of pictorial semiotic systems. Linguistic body language is, of course, tri-stratal, but the others are bi-stratal (no grammar, just content and expression).

Postscript (17/1/19):

Martin's most recent publication misunderstanding, misrepresenting and rebranding my ideas, Embodied meaning: a systemic functional perspective on paralanguage (2019), is being critiqued here.

The Intellectual Origins Of Van Leeuwen's Ideal–Real Verticality: The Medieval Scale Of Being

Kœstler (1959/79: 97-9):

It was a walled-in universe like a walled-in medieval town.  In the centre lies the earth, dark, heavy and corrupt, surrounded by the concentric spheres of the moon, sun, planets, and stars in an ascending order of perfection, up to the sphere of the primum mobile, and beyond that the Empyrean dwelling of God.

But in the hierarchies of values, which is attached to this hierarchy of space, the original simple division into sub-lunary and supra-lunary regions has now yielded to an infinite number of sub-divisions.  The original, basic difference between coarse, earthly mutability and ethereal permanence is maintained; but both regions are sub-divided in such a manner that the result is a continuous ladder, or graded scale, which stretches from God down to the lowliest form of existence. …

The […] theory was put into a more specifically Christian shape in The Celestial Hierarchy and The Ecclesiastical Hierarchy by the second most influential among the Neoplatonists, known as the pseudo-Dionysius. … It was he who provided the upper reaches of the ladder with a fixed hierarchy of angels, which were afterwards attached to the star-spheres to keep them in motion: the Seraphim turning the Primum Mobile, the Cherubim the sphere of fixed stars, the Thrones the sphere of Saturn; the Dominations, Virtues, and Powers the spheres of Jupiter, Mars, and the sun; the Principalities and Archangels the spheres of Venus and Mercury, while the lower angels look after the moon.

If the upper half of the ladder was Platonic in origin, the lower rungs were provided by Aristotelian biology, which was rediscovered around A.D. 1200.  Particularly important became his 'principle of continuity' between apparently divided realms of nature…

The 'principle of continuity' made it not only possible to arrange all living beings into a hierarchy according to criteria such as 'degrees of perfection', 'powers of soul' or 'realisation of potentialities' (which, of course, were never exactly defined).  It also made it possible to connect the two halves of the chain — the sub-lunary and the celestial — into a single continuous one, without denying the essential difference between them. …

The chain, thus unified, now reached from God's throne down to the meanest worm.  It was further extended downward through the four elements into inanimate nature. … A further downward extension led into the conic cavity in the earth, around whose narrowing slopes the nine hierarchies of devils were arranged in circles, duplicating the nine heavenly spheres; Lucifer, occupying the apex of the cone in the precise centre of the earth, marked the bitter end of the chain.

The medieval universe, as a modern scholar remarked, is thus not really geocentric, but 'diabolocentric'.

Note that, in terms of metafunction, the ideal/real distinction is, of course, an ideational distinction, not a textual one.

Van Leeuwen's 'Ideal vs Real' Distinction In Visual Semiotics

Leaving aside their interpersonal function of valuation, the meanings 'ideal' and 'real', like the meanings 'idea' and 'thing', are construals of experience, and, as such, are ideational, not textual, with regard to metafunction.

Halliday & Matthiessen (1999: 7):

The ideational metafunction is concerned with construing experience — it is language as a theory of reality, as a resource for reflecting on the world.

'Real' refers to the material order of experience: things (and other phenomena); whereas 'ideal' refers to the semiotic order: ideas of things (and other phenomena). 

As levels of abstraction, 'ideal' is higher than 'real', and positioning expressions of 'the ideal' above expressions of 'the real' in images is a visual means of representing that ideational relation.

The textual metafunction, on the other hand, is concerned with organising the ideational and interpersonal, as through giving different degrees of textual prominence — e.g. thematic or informational — to ideational and to interpersonal meanings.

Halliday & Matthiessen (1999: 7-8):

The textual metafunction is an enabling one; it is concerned with organising ideational and interpersonal meaning as discourse — as meaning that is contextualised and shared. But this does not mean processing some pre-existing body of information; rather it is the ongoing creation of a semiotic realm of reality.

If it is true that the top portion of an image has a textual function in the construction of an image as message, this is quite distinct from the types of ideational meaning that are likely to be given textual prominence by such means.

Halliday & Matthiessen (2004: 85): 

Initial position in the English clause is meaningful in the construction of the clause as message; specifically, it has a thematic function.

Van Leeuwen's Verticality As "Ideal – Real"

ideal:	upper class	powerful	Netherlands	tall	shallow
|	|	|	|	|	|
real:	lower class	powerless	Australia	short	deep