How To Use Emotion Metrics To Make Your Videos Succeed
What makes the difference between a good video and a bad video? And why isn’t there any way to measure a video, objectively, before its release?
Quiznos released an ad using the Spongmonkeys, characters from a popular internet video to promote their sandwiches. The characters quote the original video and praise Quiznos for “being good to us” (original lyric — “close to us”). The original video has 800,000 views and was shared and chopped up and meme-ified for a probably total viewership of several millions. The Quiznos ad capped at a mere 200,000 views. Why didn’t the character placement work?
The famous “Got Milk?” ad from 1993 features a character who is rendered inarticulate from a peanut butter sandwich because his tongue is stuck to the roof of his mouth and thereby prevented from answering a $10,000 radio trivia question correctly. If only he had milk to wash the peanut butter out.
Both ads took many hours of work writing and editing, and both have the same goal of selling their product. The Quiznos creative team had a good concept in attempting to capitalize on a hit internet video, but by all metrics, the effort backfired. The ad frequently appears in ‘Top 10 Worst Commercials’ lists, including Time Magazine’s ‘Top 10 Creepiest Product Mascots’ list, and the Got Milk ad frequently ranks as one of the best. What went wrong for the Quiznos video, and what went right for the Got Milk ad?
The Quiznos ad was never going to be as good as the Got Milk video, but I’m sure the Quiznos executives and marketing team would have loved to be able to dodge that bullet in terms of bad quality and wasted marketing dollars when they could have pursued better options. Similarly, the Got Milk production team probably would have loved to know the caliber of the video before they released it. Executives of both companies probably used focus groups and qualitative surveys to evaluate the ads before their release, but there have been many noted problems with these methods.
If you make videos, you’ve probably wished at some point that there was a way to test how people would react before you show people and share it. What if there were a way to measure engagement objectively before a video’s release? What if there were a way to measure a video’s likelihood for success? The old ways to test your videos, including focus groups and surveys, are outdated and obsolete; they don’t provide accurate data. Measuring emotions is the key to solving this problem.
Testing the Effectiveness of Videos Before Release
In the marketplace of experiences, where attention is currency, the ability to measure people’s attention is a gold mine.
Sticky Video Emotion Analytics Can Help
The newest addition to Sticky’s predictive platform measures the viewer’s emotional engagement and attention to track the entire experience of watching visual content.
How it Works
Sticky Video Emotion Analytics works by using opt-in participants to watch your media and tracking their micro-expressions and facial movements through their webcams to pinpoint emotional activity in response to particular frames and elements in your media.
Used in conjunction with eye tracking, you’ll be able to tell not just what people are feeling as they watch the video, but exactly what they’re looking at on-screen when they feel that emotion. If someone feels happy one second and sad a few seconds later, you’ll be able to tell exactly what person, character, logo, music or other element causes the happiness and the sadness and what changed on-screen to elicit the emotional shift.
Verification
We tested the system on the research of psychologist Paul Ekman, who discovered that emotions are universally expressed and recognized through the same facial expressions in all human populations, regardless of things like geography and culture. The emotions expressed include disgust, fear, joy, sadness, and surprise.
In this verification study, we showed participants pictures of facial expressions from Paul Ekman’s original studies and told them to express the pictured emotion. In each case, the system recognized the emotion that it was supposed to recognize. The Video Emotion Analytics graph peaked with the correct emotion in response to the facial expression displayed.
Joy
Sadness
Fear
Surprise
Disgust
The colors on the graph correspond to the various emotions, and what’s pictured above the graph is the face participants are told to mimic. As you can see, the different emotions peak as the pictures change; Sticky Video Emotion Analytics accurately analyzes people’s facial expressions and shows the correct emotion in the graph.
It is important to mention that asking people to mimic facial expressions yields less natural expressions than organic emotion does. It is hard to contort one’s face on command to replicate a shown image. However, people naturally make these faces when they feel emotions, without conscious thought or awareness. And if the system can recognize the contorted faces in this case, it will be able to recognize the more natural faces people make when actually feeling emotion.
If you want to learn more about why people broadcast their emotions on their faces, read another of my articles here.
But before we discuss how to use this data to improve your videos, we first need to understand more about emotions.
Back Up — What Are Emotions?
Basic Overview
In the most basic sense, emotions are guides to behavior when presented with a new stimuli or situation. In simplistic terms, emotions can be broken down into three categories: I like this stimuli, I don’t like this stimuli, and I want to learn more about this stimuli. Let’s take a look at what emotions fit into each category.
“There’s something new! I don’t know how I feel about it!”
Surprise — This new thing is not what I was originally expecting or looking for.
“I like this stimuli!”
Joy — I want to engage with this new thing more.
“I don’t like this stimuli!”
Fear — This new thing is threatening.
Disgust — This new thing is physically or morally repulsive.
While most emotions can be correctly placed into one of those three categories, one emotion behaves like a hybrid between “I like this stimuli!” and “I don’t like this stimuli!” That emotion is Sadness and the behavior that it elicits manifests at first as seeking and then as avoidance. We’ll get into more detail about that later.
With the basics out of the way, let’s measure the emotion data for each video. And if you want a deeper dive into the evolutionary psychology of each emotion, read a short piece I wrote on the topic here.
Case Study: Bad Video
First, let’s examine the Quiznos’ Spongmonkeys ad.
In examining a videos’ results, the first step is to watch it all the way through to see if any trends emerge from the emotions results graph.
The Big Picture
People start with a high engagement of Surprise, but that quickly shifts to Disgust.
Let’s see what causes the Disgust response using eye tracking.
Shift from Surprise to Disgust
When the Spongmonkeys are outside of the restaurant, Surprise beats Disgust.
But a second later, something changes, and Disgust beats Surprise. What happens to elicit that emotional shift?
This is the first presentation of food throughout the entire video. People are looking at the sandwich (blue heatmap) during the inflection point when the emotional shift occurs. This indicates that even though people are surprised by the Spongmonkey characters, there’s a visceral response to relating the characters to food. This is probably because food indicates breaking the body envelope, and people don’t want the Spongmonkeys to break the body envelope.
Indeed, Disgust is the top emotion for the next five seconds, a period in which a sandwich is prominent on screen.
However, as soon as the Spongmonkeys leave the restaurant, Surprise beats Disgust again.
But, Don’t Zoom in Too Much…
Zooming into one of the Spongmonkey’s grinning face leads to a peak of Disgust.
Lingering Residual Disgust
A close-up on a Quiznos sandwich (no Spongmonkeys in sight) yields a period of high disgust
This is probably on account of lingering emotional association with the food and the Spongmonkeys.
Joy With Branding Moment
Despite all the Disgust, people still feel joy on seeing Quiznos’ logo.
However, this is the first time Joy peaks, and it doesn’t discount all of the Disgust earlier in the video. The Joy at seeing Quiznos’ logo isn’t enough to save the ad.
What Do The Results Signify?
Using Evolutionary Psychology and Emotion Data to Predict Behavior
Joy
In contrast to negative emotions, positive emotions seldom occur in response to life-threatening situations. Therefore, joy and other positive emotions did not need to elicit focused attention and focused responses. Joy widens the mind, creating flexible responses, broadening the array of possible thoughts and options. Joy makes people want to push the limits, play and be creative in terms of physical, social, intellectual, and artistic tendencies.
This manifests in a variety of ways. An eye tracking study found that positive emotions lead to more attention on the periphery and wide visual search patterns (Wadlinger et al., 2006). People experiencing joy tend to be more inclusive, receptive to new information, and perform better on tests of creativity (Cohn and Frederickson, 2008).
These are all specific examples of a bigger behavioral pattern that build enduring personal resources. This phenomenon is called the broaden-and-build theory, that holds that the creative and open thinking elicited by joy allows people to broaden the scope for their environment and build out in the environment.
The takeaway from this ‘broadening’ is that joyful people are more open to choice and may be willing to try a brand they haven’t yet tried. Kahn and Isen’s classic experiment on positive affect and choice showed that people with a mild positive affect facilitates variety-seeking among different consumer brands (Kahn and Isen, 1993).
Therefore, videos or ads featuring a new brand or product should strive to elicit joy.
Sadness
Since sadness is associated with loss, behavior manifests like loss as well. From an evolutionary perspective, sadness occurred when an individual was separated from the group — lost. I mentioned earlier that sadness can be seen as a hybrid between seeking and avoidance, and now I’ll go into more detail on that point. Picture an animal separated from its mother. You would expect the animal would initially look for its mother itself. After not finding her, it would withdraw (Solms and Turnbull, 2002). Every child separated from their mother in the mall knows it’s easier to be found if you stay still, even though the natural instinct is to seek in that situation.
This behavioral pattern explains a few things about the emotion. Light sadness experienced for a short period of time can actually increase motivation. Experimenters showed participants either a happy film or a sad film and then gave them an untimed cognitive abilities test to take. Since the test was untimed, the amount of time people spent on the test was indicative of perseverance and motivation. People who were shown the sad film spent 1.5x as long on the test, and got 1.5x as many questions correct as people who were shown the happy film (Forgas, 2016). This is the seeking phase of the emotion. It is also widely accepted that when people are extremely sad or experience sadness for long periods of time, motivation can be hard; this is avoidance.
Because sadness is a loss-prevention emotion, people who are experiencing sadness have a higher buying price for objects they desire, as Jennifer Lerner’s famous ‘Heart strings and purse strings’ experiment’ shows (Lerner, 2004). In simplistic evolutionary psychology terms, this is because people experiencing sadness want to make up for a loss by acquiring more goods.
People often view sad as a ‘bad’ emotion, but a light sadness for a limited period of time can be effective for people who make videos. Video ads can make people sad in order to increase motivation and willingness to pay, but just not *too* sad lest they decrease motivation.
Fear
Fear is a powerful motivator, changing priority weightings for goals and motivations by making safety the highest priority. Focus widens to find and avoid the danger, and long-term planning is near impossible. Fear can be thought of as both a seeking and an avoidance emotion, depending on the circumstances; this duality has been famously codified as the “fight or flight” response. Fear with a solution elicits the ‘fight’ response and subsequent approach behavior, whereas fear without a solution the ‘flight’ response and subsequent avoidance behavior.
Even though fear widens a person’s field of vision to scan for potential dangers, a study on emotions and attention found that fear makes people less distractible. People experiencing fear were more likely to notice a second stimuli (Vermeulen, 2009). Therefore, fear can be used effectively to capture attention; however, a solution must be presented or else people will avoid rather than seek the stimulus. The best way to leverage fear in creating visual content is to present a specific danger and a specific solution.
In the case of specific dangers, fear increases purchase intent only if the product is perceived to reduce the perceptions of danger (McDaniels et al., 1984). However, the risk in using fear marketing tactics is the potential, through nonspecific messaging, to elicit the flight instead of the fight response.
It is also worth noting that fear activates specialized learning systems. When people are afraid, they associate everything in their surroundings with that emotion, and they remember the specific sequences of events that led to that emotion. This is why fear plays especially well with educational efforts, à la Mothers Against Drunk Driving and anti-smoking campaigns.
Surprise
As you can probably assume, surprise is good at capturing attention.
In his article “Surprise: A shortcut for attention?”, Pierre Baldi makes a mathematical argument for surprise’s importance in capturing attention. Baldi argues that while searching for information, there are competitive inputs at play: Those that are relevant to the objective of the search, and those that are unexpected or surprising while searching. Using a bayesian approach and after several mathematical models and equations, Baldi comes to the conclusion that surprise can act as a shortcut to relevance but not a perfect substitute. In other words, surprise takes precedence over relevance in the short term.
Surprise can therefore be seen as the best emotional cue for capturing attention in the short term, especially in the case of irrelevant information (pre-roll video ads that play over what you wanted/expected to watch).
Surprise is also very useful in building an object’s shareability. Tania Luna and LeeAnn Renninger put forth the following model of a four-step process a person undergoes as they experience surprise in their 2015 book Surprise: Embrace the Unpredictable and Engineer the Unexpected: 1) Freeze, when the unexpected stops us in our tracks. 2) Find, when we attempt to explain what’s going on. 3) Shift, when we change our perspective based on the new findings. 4) Share, when we feel the need to share our surprise (and the object of our surprise) with others.
Surprise is therefore also the best emotion for shareability and for measuring a video’s potential to go viral.
Disgust
Disgust is the most basic avoidance emotion, triggering strong goals to expel the stimuli. Because of this, disgust should be avoided for most videos.
In the same ‘Heart strings and purse strings’ article mentioned in the sadness discussion, Lerner discovered that disgust decreases buying and selling prices (Lerner, 2004). And in a different study on emotions and attention, researchers discovered that disgust makes people more distractible; people experiencing disgust were less likely to notice a second stimuli (Vermeulen, 2009). The authors theorized that this is because the disgust response reduces sensory exposure. Therefore, disgust is the kryptonite for any video trying to capture attention or sell something, as it lowers attention and makes the object unpalatable and undesired. Video marketers should avoid the emotion at all costs.
Unless you’re making a comedy video, avoid disgust because it can be a strong avoidance signal to your viewers.
What Predicts Success and Failure in a Video
It’s important to keep in mind that no single emotion can predict success; instead, lack of emotion is a good predictor of failure. Large expressions of disgust is also a good predictor of failure.
Let’s use the evolutionary psychology of emotion to dissect the Quiznos ad in more detail, to see what exactly went wrong and what could have gone better.
The obvious solution would be to pick a different product mascot for Quiznos, but that might not be necessary. The Spongmonkeys themselves only elicited disgust on the closeup of one of their grinning faces:
Besides that instance, the Spongmonkeys elicited Disgust when placed close to food:
The Spongmonkeys were great at eliciting Surprise, especially early on in the video, which is key today in the age of pre-roll video ads that play before many videos. This first peak of Surprise would probably stop many people from skipping the video.
I expect that if the Spongmonkeys weren’t displayed near food, they would not elicit Disgust, and the standalone sandwich later on would not elicit Disgust either, since this was probably only because of residual association from seeing the characters on the sandwich.
And in that case, the Joy from seeing the logo at the end would probably be more effective, since it wouldn’t be in addition to all the Disgust earlier in the video, and the video would have been more effective overall.
The Quiznos ad had a good concept, and the content didn’t need to be scrapped altogether — — just rearranged.
Let’s see how the Quiznos ad compares to the Got Milk ad.
Case Study: Good Video
Again, the first step is to watch it all the way through to see if any trends emerge from the emotions results graph.
The trend for this video is a near-perfect mirror image of Sadness and Joy from the start of the video to the end.
Let’s see what causes the shift from Sadness to Joy using eye tracking.
Sad Music and Large Empty Rooms Set Ambience
The video begins with images of large empty rooms (presumably a museum) full of Alexander Hamilton artifacts and memorabilia; sad music plays concurrently.
The video probably owes its sad start to the ambience. Sadness peaks again as the camera zooms on a man spreading peanut butter on bread with a knife; the knife is threateningly in the center of the frame.
The early emotional peak is effective. Remember Sadness is not a negative emotion, as it can increase willingness to pay and loss-aversion motivations. Additionally, internal studies indicate that any emotional engagement can stop people from skipping pre-roll videos, although Surprise does this best.
The ambience also sets the scene with important Hamilton-related details that come into play later, and more importantly, establish the man spreading peanut butter as a Hamilton expert.
Sadness Dips With Music and Voice
One of the first sharp dips of Sadness occurs with the music; when the speed and lightness suggest play, the emotion decreases and plateaus at a lower level.
When the music ends and the Radio DJ starts talking, Sadness levels decrease again.
Joy Overtakes Sadness
The biggest inflection point in the video is when Joy overtakes Sadness. What happens on-screen when this moment occurs?
The Radio DJ asks a trivia question for $10,000 dollars: Who killed Alexander Hamilton? And the camera shifts to two guns pointed at each other, presumably the same guns from Hamilton’s famous duel with Burr. The viewer assumes this will be easy money for the character.
Joy remains the top emotion while the camera spans across the room, from a glass encased black orb labeled “The Bullet,” to replica costumes facing each other, to a painting of the duel with Aaron Burr’s name labelled. The character then picks up the phone looking excited.
Sadness (Briefly) Overtakes Joy Again
Trouble talking with peanut butter stuck to the roof of his mouth makes Sadness rise again; however, these peaks are small and don’t last long.
Joy Rises and Rises
Reaching for a glass of milk and finding it empty causes the character to dramatically (and with a muffled, stuck-together mouth) scream “No!” while shaking the empty milk carton. This scene immediately precedes the first plateau on Joy’s rise throughout the remainder of the video.
Joy’s second peak comes as the famous phrase flashes and a narrator repeats it: “Got Milk?”
The viewer’s emotional shift from Sadness to Joy causes a perfect storm of loss prevention and seeking behaviors, which come together to increase purchase intent. This ad was so good because of the two contradictory emotions it elicits.
How to Test Your Video
Step One: Upload a File or Submit a Link
Step Two: Mark Your Emotional Goal
Step Three: Wait for the Results!
Other Research and Future Plans
While this was just a theory piece connecting the psychology of emotion to a new video analytics tool that measures emotion, we tested if this tool can be used to make videos better, and you can read about those results here. Sign up for the platform for free and try it out yourself here.
We have plans to try to make our own video and perfect it with Sticky Video Analytics. Follow us on Twitter or follow me on Medium to track our experiences trying to make a viral video!
Works Cited
Baldi, P. (2005). Surprise: A shortcut for attention.
Bonnano, G. et al (2008). Sadness and Grief. In M. Lewis, J. M. Haviland-Jones & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 797–810.) NY: Guilford.M. Lewis, J.M.
Cohn, M. & Frederickson, B. (2008). Positive Emotions. In M. Lewis, J. M. Haviland-Jones & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 777–797.) NY: Guilford.M. Lewis, J.M.
Ekman, P. (1985). Telling lies. New York: Norton
Fridlund, A. (1994). Human facial expression: An evolutionary view. San Diego, CA: Academic Press.
Forgas, J. P. (2016). Can Sadness Be Good for You?. Australian Psychologist.
Kahn, B. E., & Isen, A. M. (1993). The influence of positive affect on variety seeking among safe, enjoyable products. Journal of Consumer Research,20(2), 257–270.
Lerner, J. S., Small, D. A., & Loewenstein, G. (2004). Heart strings and purse strings carryover effects of emotions on economic decisions. Psychological science, 15(5), 337–341.
Luna, Tania, and LeeAnn Renninger. Surprise: Embrace the Unpredictable and Engineer the Unexpected. New York: TarcherPerigree, 2015. Print.
Matsumoto, D. et al (2008). Facial expressions of emotion. In M. Lewis, J. M. Haviland-Jones & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 211–248.) NY: Guilford.M. Lewis, J.M.
McDaniel, S. W., & Zeithaml, V. A. (1984). The effect of fear on purchase intentions. Psychology & Marketing, 1(3‐4), 73–82.
Meyer, Wulf-Uwe, Rainer Reisenzein, and Achim Schützwohl. “Toward a process analysis of emotions: The case of surprise.” Motivation and Emotion. 21.3 (1997): 251–274.
Öhman, A., & Mineka, S. (2003). The malicious serpent snakes as a prototypical stimulus for an evolved module of fear. Current directions in psychological science, 12(1), 5–9.
Ohman, A. (2008). Fear and Anxiety Overlaps and Dissociations. In M. Lewis, J. M. Haviland-Jones & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 709–730.) NY: Guilford. M. Lewis, J.M.
Rozin, P., Haidt, J., McCauley, C., (2008) Disgust. In M. Lewis, J. M. Haviland-Jones & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 757–777.) NY: Guilford. M. Lewis, J.M.
Solms, M., & Turnbull, O. (2002). The brain and the inner world: An introduction to the neuroscience of subjective experience. New York: Other Press.
Storbeck, J., & Clore, G. (2005). With sadness comes accuracy; With happiness, false memory: Mood and the false memory effect. Psychological Science, 16(10), 785–791.
Tooby, J. & Cosmides, L. (2008). The evolutionary psychology of the emotions and their relationship to internal regulatory variables. In M. Lewis, J. M. Haviland-Jones & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 114–137.) NY: Guilford.M. Lewis, J.M.
Vermeulen, N., Godefroid, J., & Mermillod, M. (2009). Emotional modulation of attention: fear increases but disgust reduces the attentional blink. PLoS One, 4(11), e7924.
Wadlinger, H. A., & Isaacowitz, D. M. (2006). Positive mood broadens visual attention to positive stimuli. Motivation and Emotion
Winkielman, P., Berridge, K. C., & Wilbarger, J. L. (2005). Unconscious affective reactions to masked happy versus angry faces influence consumption behavior and judgments of value. Personality and Social Psychology Bulletin, 31(1), 121–135.
