The Power of Sound and Music In Digital Storytelling
Although its impact is often overlooked, sound and music play a powerful role in shaping our experiences with digital stories. The interplay between narrative, sound effects and music are key to our perception of any form of storytelling. Think about listening to a baseball game on the radio — without any pictures you are able to vividly imagine the entire scene. In gaming, an example such as Papa Sangre (a video game with no video) shows us how sound can be used as an element to guide users through an experience with limited use of their other senses. Whether you are watching or creating a film, a game, or teaching/studying digital storytelling in a classroom setting, it is important to recognize that sound is a communication tool, which can transform our reactions in an instant.
Through three examples, I will focus on the SIPI theme of diversity and inclusion. My aim is to shed light on the ways in which sound and music can be used as a digital element to enhance storytelling for educational purposes. This piece intends to be both a teaching tool for, and example of the ways in which sound can be used to instill democratic and social awareness in student and public audiences. Moreover, I will highlight the ability of music and sound to enhance deep learning in subjects such as history, political science and literature.
“Bringing literature to life”
A Visit From Kendrick Lamar — Best Day Of School Ever? (NPR)
When New Jersey high school teacher, Brian Mooney decided to use the lyrics of Kendrick Lamar’s To Pimp a Butterfly to illustrate the messages in Toni Morrison’s novel, The Bluest Eye, he did not realize the impact he would ultimately have on his students. The book is about a young black girl who yearns to have blue eyes, and through Kendrick’s lyrics, Mooney was able to help his students develop a better understanding of the themes in the novel. Mooney was so excited by the connection he had made with his students, he wrote a blog post discussing how Kendrick’s lyrics had helped his class develop a deeper understanding of topics such as African American culture, diversity and inclusion, racism and sexism. In his post, Mooney touched on the changing nature of literature in education and the need to bring different types of learning to classrooms, which are becoming increasingly diverse. By incorporating Kendrick’s songs into his lessons, Mooney was able to bring the book to life for his students in a way that traditional methods of study could not. Mooney’s blog post ended up reaching the famous hip-hop artists, and after reading the post, Lamar was so excited by the concept; he came to visit the students and became an active participant in the class. Although Mooney’s class was called “hip-hop literature”, which gave him the opportunity and freedom to incorporate hip-hop music into his study, there are important lessons for any student or teacher to be taken from this example. First, music and sound have the ability to help students connect with, and understand social and political themes, which are often difficult to comprehend. Second, it is important to recognize the ways in which music and art can be used to drive engagement for students in traditionally dry subjects. Lastly, giving students an opportunity to develop projects drawing on the themes discussed in the class allowed them to use their own creativity, which resulted in a learning environment where the diverse talents of their group could be showcased and appreciated.
“Music & Mental health”
Hip-hop helping mental health patients to heal (CTV News)
Music and sound can also be used as a tool to help people struggling with mental health issues. In addition to helping teachers and students explore topics and themes in literature, music and sound also have the ability to bring new meaning and understanding to the lives and struggles of individuals. A pilot project in Victoria, British Columbia is using a hip-hop therapy program to help youth coping with mental health issues to heal. During the process of creation, which includes speaking about their issues and expressing their ideas through music, students are able to write and share original music, helping them to better understand themselves, and feel more confident around others. Hip-hop music in itself is a form of storytelling rooted in self-discovery and overcoming personal challenges. Through this unconventional form of therapy, the youth are developing a better understanding of their own story and struggle. By giving the students the creative freedom to develop new connections within their own minds, music is allowing the students to actively engage in learning about themselves in new ways. Moreover, the opportunity to share their stories with other people is bringing them higher self-esteem, and in turn a higher quality of life. With regards to SIPI, this story highlights the theme of diversity and inclusion, and the ways in which a digital tool such as sound and music can be used to help people dealing with mental illness better understand themselves, others and the world in which they live.
“Music and the Holocaust”
This example will explore how music and sound can be used as a teaching tool, which can evoke powerful emotions. Music and sound have the ability to convey a feeling or message in ways that words are often unable to capture. Even pieces written without lyrics can convey deep meaning and emotion to an audience. Through abstract, free association listeners are able to develop their own interpretation of the meaning of a piece, and put together a story or mental image based on how the music makes them feel. Listening to ambient music, which has been written to depict a specific feeling or historical event, can be a powerful teaching tool in the digital storytelling arsenal. Music and sound can be used as a tool for digital storytelling, assisting practitioners in bringing subjects such as history and political science to life. Music also affords the ability to discuss important social issues such as mental health, race, gender, religion and sexuality in a more accessible fashion.
In a piece entitled “Music of the Holocaust: A thematic design for string music education” from Lisa C. Delorenzo’s Giving Voice to Democracy in Music Education: Diversity and Social Justice, Tamara Reps Freeman states:
“Middle school and high school string instrument classes and ensembles can bring to life the lessons and legacies of the European Holocaust, due to the wealth of extant archival folk music and instrumental music composed by the Jewish people interned in the World War II ghettos and concentration camps. Although the Third Reich was imposing fascism throughout Europe from 1933 to 1945, music provided a desperately needed democratic voice for Jewish victims. Evocative music captured and responded to the events and emotions of the Holocaust through expressive melodies and graphic lyrics. Ironically these defiant pieces were freely sung and played, often to the naïve delight of the Nazis who did not understand the subliminal musical messages of spiritual resistance.”
In a three-piece movement for a string quartet entitled “Different Trains” (available below), renowned minimalist composer, Steve Reich expressively illustrates his sonic interpretation of three distinct points of the Jewish experience in World War II. Written in 1989, Reich wrote the piece as a reflection on his experience of traveling on a train from New York To L.A. during the war. The piece is intended to convey that as a Jew, traveling on a train in Europe at the same point in time would have likely resulted in a very different fate.
Conclusion
Diversity and inclusion are important themes in both the study, and practice of digital storytelling. As discussed in this project, sound and music can play an integral role in shaping the emotional experience of teaching and learning these important subjects. Regardless of the medium used to depict a digital story (games, online/VR experiences, or traditional methods of study with music as digital enhancement), the power of sound and music cannot be understated. With regards to games, a new documentary entitled “Beep” explores the history of sound in gaming. In addition, this interview with Brad Fuller, the former audio director at Atari games exemplifies the important role sound plays in digital storytelling, and gaming in particular. Although it is important to recognize the theory behind using sound effectively in digital storytelling, we must not overlook the power of sound and music to emotionally engage an audience and support the themes of a narrative in both traditional and digital storytelling.