Perspectives on Multimodality (Document Design Companion Series)

Table of contents......Page 6
Acknowledgements......Page 10
Introduction......Page 12
Multimodal issues......Page 18
1.. Multimodality - the late discovery of the obvious......Page 20
2.. How many modes is `multi'?......Page 22
3.1. Semiotic properties......Page 27
3.3. Semantic potential......Page 28
4.. Mode integration: Overlapping and mixing......Page 29
5.1. The typographic image......Page 30
5.2. The verbal image and the language-image-link......Page 32
6.. Semiotic principles across modes......Page 35
Notes......Page 38
References......Page 40
1.. Introduction......Page 42
2.. Language vs. semiotic systems......Page 43
2.1. Stratal organisation and grammar......Page 46
2.2. Semiotic systems and metafunctions......Page 49
3.1. Mode in SFL......Page 51
3.2. Mode as the primary activator of semiotic systems......Page 52
3.3. Fractal principles between semiotic systems......Page 55
4.. Conclusion: An example using gesture......Page 57
References......Page 59
Notation conventions......Page 61
1.. Multimodal semiotic resources......Page 62
2.. Nature of a semiotic resource......Page 63
3.. Visual images as semiotic resources......Page 65
4.. Extending the claim of arbitrariness......Page 68
5.. Basic building blocks: Words and icons......Page 71
Note......Page 72
References......Page 73
1.. Introduction......Page 76
2.. An example of interpretative analysis......Page 77
3.1. The origin and representation of annotated corpora......Page 79
3.2. Annotation problems with complex data......Page 81
4.1. The GeM model......Page 84
4.2. Designing and populating a multimodal corpus......Page 85
5.. Examples of using a GeM-annotated corpus......Page 93
6.. Conclusions and directions for the future......Page 96
References......Page 97
Analyses and applications......Page 100
1.. Introduction......Page 102
2.. Wigner on the effectiveness of mathematics......Page 104
3.. A systemic functional approach to multisemiosis in mathematics......Page 105
4.. A historical perspective of mathematics......Page 107
5.. The Cartesian project......Page 115
6.. Newton's view of the world......Page 117
a. Language......Page 120
b. Visual images: Graphs and diagrams......Page 121
c. Mathematical symbolism......Page 122
8.. Inter-semiosis in mathematics......Page 123
9.. Complexity, chaos and Wigner on the effectiveness of mathematics......Page 124
Notes......Page 126
References......Page 127
1.. Introduction......Page 130
2.. Premises......Page 131
3.1. Pronunciation and acoustic phonetics......Page 133
3.2. Text and image......Page 134
3.3. Speech and gesture / lip movement......Page 142
Notes......Page 145
References......Page 146
Reviewed language teaching CD-ROMS......Page 147
1.. Introduction......Page 148
2.. Constructions of class......Page 150
3.. Constructions of gender......Page 151
4.. Uses of multimodal discourse analysis......Page 152
5.. Multimodal configurations and issues of genre......Page 158
6.. Results: Towards a theory of multimodality......Page 159
7.. Conclusion......Page 160
Notes......Page 161
References......Page 162
2.. Translation studies......Page 164
3.. Creative texts......Page 166
4.. Subtitling......Page 168
5.. The multimodal approach......Page 171
References......Page 181
1.. Introduction......Page 184
2.. Multimodal humour and its translation......Page 185
3.1. Linguistic word-play......Page 187
3.2. Plays on words reinforced by non-verbal signs......Page 188
3.3. Plays on signs depending on a multimodal combination......Page 189
3.4. Non-verbal plays on signs......Page 193
4.1. Culture-specific pictorial elements......Page 194
4.2. Pictorial intertextuality......Page 196
4.3. Pictorial allusions......Page 197
5.. Comic aspects of typography......Page 200
6.. Summary......Page 201
Source texts......Page 202
Secondary texts......Page 203
1.. Introduction......Page 204
2.2. Multimodality theory......Page 205
2.3. Linguistic and multimodal analyses of museums......Page 206
3.. The questions to be explored......Page 207
4.1. The floor plan of the museum......Page 208
4.2. Zones in The Dancing Master's Hall......Page 209
5.1. Linguistic analysis of the text......Page 211
5.2. Visual analysis of the family portrait......Page 214
6.. Synthesis and discussion......Page 216
References......Page 218
Floorplans and images......Page 219
Zone 2 - Processes in the original Residence text......Page 220
1.. Introduction......Page 222
2.. Paving the way for a deep-analysis of multimodality in contemporary text corpora......Page 223
3.. Text versus hypertext: Do we need a new conception of text?......Page 225
4.. Approaches to theories of inter-semiotic layering......Page 227
4.1. The model proposed by Hoek (1995)......Page 228
4.2. Practical application of the model on sample (hyper)texts......Page 229
5.. Challenges for the future: Intelligibility......Page 233
6.. Conclusion......Page 234
References......Page 235
1.. Interpreting in hospitals......Page 238
2.. Interpreting and multimodality......Page 239
3.. The use of diagrams in medical briefings for informed consent......Page 240
4.. Briefing for a gallstone operation......Page 241
4.1. Introducing the organs in the source language......Page 242
4.2. The organs in the target language......Page 243
4.3. The suspected diagnosis in the source language......Page 246
4.4. The suspected diagnosis in the target language......Page 247
5.. Conclusion......Page 248
References......Page 249
Index......Page 254