Weekend Diversion: An Optical Illusion For Your Ears

The sounds you hear are affected by what you see. Experience it for yourself!


“It is the function of art to renew our perception. What we are familiar with we cease to see. The writer shakes up the familiar scene, and, as if by magic, we see a new meaning in it.” -Anais Nin

We’re all used to trusting our senses for information about this world. And while there are certainly plenty of illusions that lead us astray, they almost always come up as tricks that our eyes play on us. Have a listen to Thievery Corporation’s song, The Supreme Illusion,

https://play.spotify.com/track/2T6WDL5LJmT85XdN6Uz1jz

while I introduce you to a whole kind of illusion that involves your ears!

Image credit: Fromkin, Victoria and Robert Rodman. An Introduction to Language. 5th ed. Fort Worth: Hartcourt Brace Jovanovich College Publishers, 1993. Via http://www2.uncp.edu/home/canada/work/caneng/phono.htm.

You all know how to pronounce the different consonants, and how to make a plethora of different sounds with your mouth. For example, if you wanted to make the “B” sound, you’d purse your lips together, and burst them outwards at the moment you wanted to create that distinctive “B”.

In fact, just by looking at a snapshot of someone’s mouth, you can often tell what the next consonant sound they make is going to be!

Image credit: screenshot from BBC Two.

Well, looking at someone’s mouth who’s about to make that “B” sound, you’d most certainly never confuse it with a sound that you make by blowing air between your upper teeth and your bottom lip, like the “F” sound. Right? I mean, it’s hard to get more different — from a consonant perspective — than those two sounds.

The “F” sound is unmistakeable, right?

Image credit: screenshot from BBC Two.

Believe it or not, if you listen to the “B” sound while you watch someone’s mouth go through the motions of an “F” sound, you will hear the “F”!

Not only will you hear the “F” sound, but you’ll even hear it even if you know it’s coming. And it’s coming.

Try it. Watch the video below.

https://www.youtube.com/watch?v=G-lN8vWm3m0

This amazing phenomenon is known as the McGurk effect, and I must have watched the above video five or six times already, because no matter how hard I try, my sight overrules my ears! With other optical illusions, I can sometimes overrule them and “see through” the illusion once I’ve figured it out, but I can’t with this one. The only way to overcome it, as far as I can tell, is to close my eyes. Only then do I always hear the true sound.

So I went to look for other examples, and look what else it works with!

https://www.youtube.com/watch?v=aFPtc8BVdJk

Do you hear the “Ga” (or maybe the “Ca” or “Da”) sound coming out of this guy’s mouth? Replay it and close your eyes, and listen again. Your brain pulls both the audio and visual stimuli together and does its best to interpret the confusing signals, but this illusion is incredibly robust!

Image credit: Esther Wiersinga-Post (edits by me), via http://www.ai.rug.nl/nl/project/integratie.html.

There are all kinds of videos of this illusion out there, but what I’m most fascinated by is that some types of brain damage to the left hemisphere, increase the effect, while other types as well as disorders such as dyslexia, autism, alzheimer’s or schizophrenia lead to reduced McGurk effects.

This is the first hearing illusion I’ve ever come across, and this is a spectacular one. (If you haven’t checked out our Comments of the Week this week, go and check it out.) Hope you enjoyed it, and have a great weekend, too!


A version of this post originally appeared on Scienceblogs. If you have a question or comment, head over there to the Starts With A Bang forum and weigh in!