Multimodality as an Educational Challenge
Guest Authors: Katarzyna Polkowska and Jan Sójka
2nd Year Students at the MA in English Philology, University of Białystok, Poland
The exponential growth of computer-mediated communication over the past two or three decades has had a significant influence on the literacy habits of children who are currently enrolled in schools. Children in industrialized countries generally prefer to interact with their classmates using electronic means, such as texting, messaging applications, or any other program that can facilitate communication.
Because the abilities that are utilized in the classroom of a language are closely associated with the workplace of the 21st century, equitable access to computers and multimodal literacy are essential components of a modern language school. In his massive study on how technology has altered society, Castells (2000) states, “Among information technologies, I include, like everyone else, the converging set of technologies in microelectronics, computing (machines and software), telecommunications/broadcasting, and optoelectronics” (Castells, 2000: 29). Castells believes that the revolution brought about by information technology is on par with the industrial revolution that occurred in the 18th century in terms of the fact that it has brought about a contrast between how people lived, worked, and exhibited their identities before and after the revolution. The integration of new technologies into educational settings inevitably results in the production of texts that frequently demand “new literacies” in contrast to “old literacies” (Luke, 2002).
These forms of literacy are distinct from those taught in a literature-based English curriculum, in which a written book and the words contained within it serve as the foundation for instruction and study. The proliferation of new forms of media and the ever-increasing requirements of today’s students are putting an unprecedented amount of pressure on educational environments of all kinds to undergo a process of reorganization and transformation. In the field of literacy pedagogy, the increasing influence of digital technologies has produced a vibrant and challenging body of theoretical and practical interest in “multimodality” — a term that refers to the practice of meaning-making encompassing the purposeful assimilation of semiotic resources including, but not limited to, writing, images, speech, gestures, drawing, and sound (Emmison& Smith, 2000; Kress, 2003; The New London Group, 1996; van Leeuwen&Jewitt, 2001).
It is vital to observe the distinction between medium and mode in this definition. Nelson (2006) notes that these concepts are frequently confused. Medium relates to the method through which technology is transmitted, such as a printed book, CD-ROM, or computer program, whereas mode relates to the text, gesture, sound, colour, and other types of signification. The concept of meaning-making is fundamental to the study of multimodality. Lemke (1998) notes that the different modes that interact with the printed text, such as sound, gesture, and so on, are not just ornamental additions to a text whose primary source of meaning is the written word. In a multimodal text, such as a website, all the modes that interact are vital to the meaning, “making the whole far greater than the sum of its parts” (Lemke, 1998: 284). Lemke argues that the most significant impact of technology on literacy is a shift away from logocentrism or an intense focus on the printed word.
Although multimodality is not wholly new, and printed texts have always been supplemented by drawings, diagrams, etc., the approach to text has been logocentric, which “has identified language alone as a reliable medium for logical thought, and written language as the primary medium of authoritative knowledge and, more recently, of all higher cognitive capacities” (Lemke, 1998: 284). Nevertheless, given that texts are becoming multimodal, it is crucial to acknowledge that the written word is only one among several modalities that express meaning. With the variety and hybridity of contemporary texts, understanding the consequences of multimodality is crucial to any theory of meaning-making (Cope &Kalantzis, 2000; Kress, 2000, 2003; Kress & van Leeuwen, 2001; Nelson, 2006).
According to Cope and Kalantzis: “Meaning is created in increasingly multimodal ways, in which written-linguistic forms of meaning coexist with visual, auditory, and spatial patterns of meaning. Consider the multimodal methods in which meanings are created on the World Wide Web, in video captioning, in interactive multimedia, in desktop publishing, and in the usage of printed texts in a shopping mall, for instance. To navigate this developing universe of meaning, a new multimodal literacy is required” (Cope &Kalantzis, 2001: paragraph 4). As a field that is just a few decades old, there is a need for research in several specific areas. For instance, there is a paucity of large-scale quantitative research with a generalizable sample that can demonstrate the links between gender, race, socioeconomic class, and technology use in language classes. Arguably, there is also a need for experimentally designed research that identifies the specific parts of language learning that children who use technology acquire but those in the control group do not.
Last but not least, there is a severe lack of research on testing multimodal literacy. How precisely does a teacher evaluate a multimodal text, and how does this style of evaluation vary from standard text evaluation? The imbalance between the use of technology in the classroom and standardized testing is related to this issue. In the majority of nations, school-level standardized testing does not make considerable use of computers; hence, instructors are hesitant to spend time integrating technology into the classroom, as doing so might disadvantage their children on exams.
Positively, a multimodal approach to literacy provides sociolinguists and language instructors with interesting and fruitful prospects, given the capabilities of new media and digital technology in general. Where available, new media are questioning and altering traditional conceptions of “text” and “reading behaviours.” The works of Jewitt (2002) and Unsworth (2006) indicate that alternative presentations of material — those that go well beyond the written word alone — expand interpretive resources such as ‘character’ and ‘plot’ to the extent that the construction of meaning from a multimodal text necessitates the established customized or individualized design-based purposes.
Teachers of foreign languages would do well to understand what multimodal literacy is (or maybe in the future) and how multimodal expression can be utilized to meet the requirements and interests of their students. Where accessible, digital technologies provide students with unparalleled access to new media and publication platforms where they may exhibit their skills.
References
Castells, M. (2000).The Rise of the Network Society (The Information Age: Economy, Society and Culture, Volume 1) (2nd ed.). Wiley-Blackwell.
Jewitt, C. (2002). The move from page to screen: the multimodal reshaping of school English. Visual Communication, 1(2), 171–195. https://doi.org/10.1177/147035720200100203
Kalantzis, M. (2015, October 8). Putting multiliteracies to the test. Academia.Edu. https://www.academia.edu/2804121/Putting_multiliteracies_to_the_test
Kress, G. (2003). Literacy in the New Media Age (Literacies).Routledge.
Leeuwen, T. V., &Jewitt, C. (2001).The Handbook of Visual Analysis (First ed.). SAGE Publications Ltd.
Luke, A. (2002). What happens to literacies old and new when they’re turned into policy. In C. Bigum, D. E. Alvermann, M. Knobel, C. Lankshear, & M. A. Peters (Eds.), Adolescents and Literacies in a Digital World (pp. 186–205). Peter Lang.
Nelson, M. E. (2006). MODE, MEANING, AND SYNAESTHESIA IN MULTIMEDIA L2 WRITING.Language Learning & Technology. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/16624a37-ed39-486a-86fe-b97778e92220/content
Reinking, D., McKenna, M. C., Labbo, L. D., &Kieffer, R. D. (Eds.). (1998). Metamedia literacy: Transforming meanings and media. In Handbook of Literacy and Technology (pp. 283–301).Taylor & Francis.
Researching the Visual — Images, Objects, Contexts & Interactions in Social & Cultural Inquiry (00) by Emmison, Michael J — Smith, Philip D [Paperback (2000)]. (2000). Sage s, Paperback(2000).
The New London Group. (1996). A Pedagogy of Multiliteracies: Designing Social Futures. Harvard Educational Review, 66(1), 60–93. https://doi.org/10.17763/haer.66.1.17370n67v22j160u
Unsworth, L. (2005). E-literature for Children: Enhancing Digital Literacy Learning (1st ed.). Routledge.
