Altered Perception
Understanding Visuals Through Sound
Have you ever cried watching a sad scene in a movie? Felt the absolute fear that characters feel in horror films? Or the joy of a family being reunited at the end of the movie? The magic of film is the ability to draw us into another reality, and an essential element of this ability is film scoring. We don’t hear triumphant music in real life when our favorite football team makes a comeback or the feel-good music that movies include when a character happily walks down the street, so why do we need music to make movies seem more realistic?
To understand the importance of music in film, we need to first understand how our brains process the information given to us visually. The key to understanding how we interpret our surroundings lies in the Kuleshov Effect.
Kuleshov Effect in action
The Kuleshov Effect is a psychological theory derived from a study conducted by Soviet filmmaker Lev Kuleshov in which he suggests that an audience’s perception of what is happening onscreen is better determined by the interaction between multiple sequential camera shots than just one shot alone.
Take a look at this photo for example:
In the study, the audience was shown a shot of a man with an emotionless face, followed by a bowl of soup, and then back to the shot of the actor. The audience was then asked to describe what the actor was feeling. The audience that saw this scene tended to respond that he was hungry.
Now, the researchers took the same shot of the man, but replaced the bowl of soup with a shot of a girl inside a coffin. This time, the audience responded that they thought the man was grieving.
In this round, they swapped out the second camera shot for one with a woman lying on a divan. The actor was perceived to have thoughts of desire.
All three of these scenes had the same emotionless face, yet each audience had dramatically different interpretations of what the man was feeling.
The key finding of this study is that the context of the scene and the sequentially edited shots can have more influence over an audience’s interpretation of characters’ emotions than simply the acting itself. Here is an example of the Kuleshov Effect being used in modern filmmaking. This is the full fight scene between Batman and Bane in The Dark Knight Rises. Look at how the directors and editors use the fight to show Catwoman’s regret for leading Batman into Bane’s trap.
If you were to string together Catwoman’s shots in this scene and watch them alone, it would be difficult to understand what she was feeling. But notice that the editors chose to strategically place her shots right after the moments Batman is seen to be weak and losing the fight — thrown to the ground or strangled rather than throwing the punches. This helps the filmmakers tell the audience that Catwoman does feel remorse.
Film scoring — providing context
Film scoring is crucial to building context around the scene. Storytellers can use music to manipulate the audiences and tell them how to feel, which makes movies more immersive for the audience. It is why you can gather what a character is feeling or where a scene is taking place just by listening to the film score. Music can even tell the audience information the character doesn’t know — creating dramatic irony. Take a look at this video for an example of how music can affect your interpretation of a scene:
In horror movies, dissonant or eerie music can hint to the audience that something bad is going to happen before the character even realizes it, which is why we often find ourselves thinking something like: “Oh no. Don’t open that door. That’s a bad idea…”
It is also why we feel differently about the dinosaurs in Jurassic Park than we normally feel about them. We first hear the theme of the movie when the travelers see dinosaurs for the first time. These species are made out to be majestic and peaceful, contradicting the terrifying characterization of the T-Rex and the raptors in the later parts of the movies.
Action movies tend to have constantly repeating rhythmic beats in the background accompanied by an ensemble of instruments that get louder and louder. The fast-paced repetition mimics the way our hearts begin to race when we feel we are in danger.
Take a listen to this piece from the movie Mission: Impossible — Rogue Nation:
Just by listening to the music, you can hear when a perceived threat or antagonist would pop up on the screen. What about the moments where the fast-paced music seems to stop all of a sudden? It is possible that the character is at a crossroads and taking a moment to pause and figure out which way to turn. Without even having watched this scene, we know that the character is in danger. The music has given us the context of the scene without any acting.
It is the job of a film composer to be able to help tell the story of the film through the music in the background. And for every score, there are two important decisions that the composers need to make — what instruments to use and what harmonies to write.
Instrumentation
Choosing the right instruments to use in a film is crucial. It is no wonder Black Panther’s film composer — Ludwig Göransson — spent a month in Africa just researching possible singers, like Baaba Maal, and instruments, such as the talking drums, to use in the film score.
Each instrument has unique characteristics, or what musicians refer to as an instrument’s timbre — the tone color or tone quality. It is why we can tell the difference between a trumpet and a violin, even if the same note is being played. The timbre of an instrument can be described with words such as warm, cold, thick, bright, or clear. The combinations of the different instruments allow composers to convey different meanings in their music.
For example, the low tones of tubas, string basses, or trombones played rough and cold can be used to characterize a villain as big and intimidating. Alternatively, a warm tone of low strings and high wind instruments can be used to introduce the unsung hero in the movie. This can be heard in the Star Wars soundtrack.
John Williams’ “Imperial March,” Darth Vader’s theme, is comprised of robust brass sounds and accompanied by the harshness of the high winds and strings for intimidation. In contrast, Yoda’s theme introduces a warm cello and peaceful sounds of a harp and flute to denote his kindness and wisdom.
Harmony and intervals
A lot of people may think that harmony is just the sweet-sounding result of what happens when notes are played simultaneously. But harmony can also be dissonant and unnerving.
Different note intervals produce different reactions. A major chord sounds sweet to our ears. A piece that stays in the major key can sound peaceful or happy. Minor keys, like the ones in the sad Pixar scenes, make us emotional. There is even a chord for horror films called the tritone — otherwise known as diabolus in musica (the devil in music). When played, this chord creates an “ugly” dissonant sound that often makes people feel uneasy.
Take a listen to this Prologue of the Get Out soundtrack by Michael Abels, which showcases the use of tritones in film music.
There is a scientific reason why different intervals elicit different reactions from us and why we classify harmonies as consonant or dissonant. This is related to the topic of tonotopy, which is the spatial mapping of the hair cells in the cochlea of our ears that process different frequencies. It is how we can distinguish between different pitches in speech and music.
Different sections of the basilar membrane vibrate in response to the different frequencies that we hear. Each note interval has a unique mathematical frequency (Hz) ratio. The simpler the integer ratio is, the easier it is for our brain to process. This is how we determine certain harmonies such as an octave or major chord, which has frequency ratios of 2:1 and 4:5:6 respectively, to be consonant. The devil’s interval and diminished chords, on the other hand, has ratios of 45:32 and 160:192:231. These more complex tones create dissonance in music.
The different combinations of harmonies can elicit different emotions, which is why the addition of music in films greatly helps immerse the audience in the story that they are watching.
Storytelling in Marketing
Understanding the way people process information can help us better convey the story we are trying to tell. In advertising, commercials allow a brief amount of time for companies to deliver their selling point. The right music can contribute a lot to a campaign — whether it urges the audience to make a purchase or promotes a certain brand image for the company.
It is no secret that The Coca-Cola Company is known for its strong marketing communication. We are all very familiar with some of Coca-Cola’s campaigns, including “Taste the Feeling” (2016). Well, in these commercials, you can also “hear the feeling”.
If you watch various Coca-Cola ads through the years, you will notice that the feel-good type of music is consistent within Coke commercials. This is because the company uses its marketing campaigns to associate its famous soda with all the warm and fuzzy feelings of love, family, friendship, happiness, unity, etc. The visuals paired with the right music can make or break a marketing team’s ability to deliver a powerful and moving advertisement.
Let’s revisit one of Coke’s most famous commercials of all time. How can you not smile watching this?
While we don’t have any quantitative information on how this commercial directly contributed to the product’s financial success, we do know that the Coca-Cola garnered a significant amount of praise as shown by the 100,000+ letters the company received and the fact that the company released the song as a single which reached the Billboard Top 10.
As companies and advertisers explore more avenues to increase their profits, creating a music playlist to be played in stores, restaurants, or even online may be something to consider. The right music may help promote a positive brand image for a company and subconsciously influence shoppers to buy more. From what we have learned from the power of music in films, we know that what we hear is just as important as what we see.
Karina Chiu is an intern at Olson Zaltman
