An Interview with Dr. Emma Rodero
Emma Rodero is a professor in the Department of Communication at Pompeu Fabra University in Barcelona, Spain. She obtained a Marie Curie fellowship to conduct research in the US on the cognitive processing of sound messages using psychophysiological techniques. She is the author of twelve books and over seventy articles on voice, radio, and sound.
KM: Radio drama scripts use a specific format that cues the listener about the environment, which characters are speaking and what actions are being done (sound effects also help with these cues). Novels are not written to cue the listener in this way. way.
- Should the full-cast audiobooks be rewritten as radio scripts to provide these cues? If not, is it harder for listeners to understand what is happening in the novel if it is read verbatim?
ER: For me, absolutely yes, because it’s sound, and we know that when you are listening to an audio, regardless of the type of audio you are listening to, you need some cues, you need some strategies to pay attention and to understand well the content. It’s not the same to read something or to listen to it. Of course, one of the main problems that we have when the voice overs read audiobooks is that they don’t have these cues in the text. They don’t have these elements which are special to oral language. It’s not oral language. In many cases, it’s very complicated to read something in this written language with no cues, with no rhetorical devices, with long, long, long, long sentences that are very complicated to process, so we need different elements when we are listening to a story and when we are reading the story. For me, it’s absolutely important.
For me the best idea is to adapt or to write the stories in a different way–to write the stories to be listened to, not to be read, which is what you find in the book. We need to change that.
It’s something that is not new. Well, the story is repeating again the same ideas and the same initiatives. When radio was born, the stories or the programs, in many cases, were adaptations. The most typical example is Orson Wells and the “War of the Worlds.” It was a book, so it’s an adaptation. Orson Welles didn’t read the book. Orson Wells made a script especially for the representation for sound. It’s not a new idea. It’s something that radio has done for many years.
KM: Radio dramas are often called “Theater of the Mind.” In Bob Leighton’s 2015 article, “Theater of the Mind Makes for Great Radio, Ads and Broadcasts,” he states: “theater of the mind is a powerful thing . . . . With radio, the listener imagines . . . and by imagining, participates . . . And their imagination isn’t limited to visuals or images — but rather extends far beyond into to feelings, emotions and all other senses.” The radio drama format enables listeners to experience immersion and transference more thoroughly.
- Do full-cast audiobooks create more immersion and transportation into the story than one-author narrated books? If so, why?
ER: Absolutely, yes. Well, as you know, we are now conducting an experiment about that. We are going to test how the different elements work–how the music works, how the sound effects work, and especially, if you have a narrator and characters–how does it work?
I have done research about that previously in my 2011 article, “Stimulating the Imagination in a Radio Story: The Role of Presentation Structure and the Degree of Involvement of the Listener.” In this paper, I tested what was the most important type of narrative that you use in an audio fiction–if the characters are acting or if there is a narrator–and especially for creating more mental images and for involvement in these cases. It was these two variables that I studied. The conclusion was very clear. When you want to create more mental images and to get the listener fully involved in the story, you have to use the characters acting and not a narrator. To create more immersion and transportation into the story, it’s better to use characters and different voices.
KM: In the AudioShaper’s 2014 article “What is Sound Design?”, they stated: “Sound design is the process of recording, acquiring, manipulating or generating audio elements . . . . Sound Designers understand the tremendous power of sound to aid the storytelling process, to transport an audience directly into the vortex of the performance and to make that performance a truly unforgettable experience.”
- How can full-cast audiobooks use sound design to improve the quality of the work?
ER: In a few months, I’m going to answer this question with an experiment that we are now conducting/running, but my hypothesis is about how the use of sound design, the use of sound effects, and the use of music in audiobooks is going to improve all the variables that we are studying. We are studying the creation of mental images. We are studying the engagement. We are studying likes and dislikes, and especially attention to the stories. My hypothesis is that if you use music and you use sound effects, along with different voices or different characters, you are going to increase all the variables and approve the experience. It’s evidence. It’s logical.
Imagine you are listening to a book that sometimes is more than 8 hours or 10 hours. When you are listening to something for a long period of time, you need some kind of elements to retain the attention of the listeners. We know that these kinds of elements to get the attention are always changes and variations in the sound. It could be a piece of music or a sound effect. Also, it could be a different shot, a different distance. For example, now I’m in the first shot talking to you; now I’m far away. You can use these differences in sound to get more attention.
KM: In your article, “See It on a Radio Story: Sound Effects and Shots to Evoked Imagery and Attention on Audio Fiction,” you state: “The main function of sound effects in a fictional story . . . is precisely to create an audio reconstruction of reality, imitating reality’s actual sounds so as to create in the listener’s mind a specific image of the phenomenon that it is intended to represent.”
- How can the use of sound effects affect the listener’s ability to imagine the environment, the characters and the actions in the audiobook story?
ER: You can use sound effects for these four functions: objective, subjective, descriptive, and narrative. I always differentiate the different functions that sound effects and music can have.
Sound effects can have what I call an objective function. For example, imagine that a character runs rapidly, and you can hear the sound of the running. This is a sound effect used with an objective function.
You can use a sound effect with an expressive or subjective function–for example, to create a sad mood or emotion. If you want to induce “sad” to the listeners, you can use sound effects of raining or maybe something nostalgic.
The third function in which you can use sound effects is descriptive. All of my studies analyze descriptive function. Sound effects are very important to identify the space in which the action is developing. For example, if I am in a parking lot, the sound of the cars can be used. If I’m in the street, the sound of the traffic can be used. This is the most typical, and it is very important to identify what I call the space dimension.
The space dimension is the space in which the characters are moving and also the objects that you have in the space. For example, on the beach, the space is the beach, so I can hear the waves. In this scene, in this environment, I can have objects. For example, the environment could be a bar, so there would be bar sound effects. Music can also be used; for example, there could be background music in the bar. Therefore, sound effects are very important to define the spatial dimension.
The last function of sound effects is narrative, which structures the narration. For example, sound effects can be used to link different parts of the narration or to differentiate the beginning, the development, and the end. For example, imagine that I finished my story with the sound of the train far away, fading out, and this is the closing sound for me.
All the functions are very important because we know that way you use sound effects to create all the perceptions of the listener make it more life-like, and this is important to be transported to the scene. If I don’t perceive that the story is real or is life-like for me, I’m not going to feel inside. I’m not going to get this immersion, so it’s important to get this feeling with sound effects.
KM: In your 2017 ABMA BVAM presentation on Video & Audio Tell a Vision, you explain how intonation, speech rate, high/low pitch and clear/rough voice, word accent and rhythm all affect the listeners’ attention (prosody), comprehension.
- Why should voice actors in full-cast audiobooks learn the above principles and techniques?
ER: Audiobooks are the most difficult product that voice-over artists have to make because there are a lot of hours in front of the microphone. Audiobooks are very long, and in many cases, 8 hours or10 hours, so they need a very accurate vocal technique to support all these hours of talking. If the voice-overs are not professionals, and they don’t have good vocal techniques, they are not able to do it because they probably won’t have a voice in the second week. It’s very demanding task.
The second thing for voice overs, from the point of view of the listener, it’s very important that voice actors or voice-overs do a good job. The listener is hearing the narrator for hours and hours, so it’s very important to know how to narrate. There are not many voice-overs that master these abilities–to narrate in a good way, in a natural way. There are many voiceovers that read the book in a very monotonous way and repeat the same patterns throughout the book. It’s very hard to keep the listeners’ attention this way. It’s very important to train voice-overs in intonation, in speech rate, word accent and rhythm to get the listener’s attention and comprehension.
There is a debate about this aspect. There are a lot of people who think–inside the industry of the audio books–that the narrators have to read exactly word-for-word what the writer wrote using a neutral and flat intonation. If the narrators change their voices and try to be natural and to give a sense of what they are saying when they are reading, they are changing the original work of the writer because they are interpreting the book, and they shouldn’t interpret anything. They have to respect the original work of the writer avoiding any interpretation. The problem here is that this is not a book with written words. It is an audio with a voice reading. And voices make sense when they talk. It is life. The most unnatural thing you can do is to read the whole text, regardless the different situations and emotions, with the same robotic and monotonous way breaking the rules of oral language.
It’s the same with music and sound effects. There are many people saying that if you use music or sound effects, you are modifying the original work of the writer, and you cannot do that. Of course, I don’t agree with all these opinions, but there are many people that are thinking this way, so now there is a debate about that. Audiobooks are audiobooks, and you just have to read the book and that’s it.
On the contrary, I think that Audiobooks are audio, and, because of that, they have to follow the rules and to the strategies of oral and audio language to get the listener’s attention.