Why We’re So Easily Duped
And what good design can do about it
I remember the first time I experienced “mind control.”
It was a small conference in Virginia, and the presenters were talking about modern uses of classical conditioning (think Pavlov’s dogs). During their presentation, they led us through an exercise during which we would write down a thought on a notecard, and then pass it around the table when instructed. The presenters played a “ding” sound whenever we needed to pass along the paper, and sure enough, we quickly began passing the notecards upon only hearing the “ding,” without any further instruction. They classically conditioned us to pass notecards at the “ding” in a matter of minutes, entirely without our realization.
We were told that while this was a very simple example of “mind control,” large corporate organizations often use similarly subtle tactics to influence consumers’ every move, and we should be aware of the dangers of this persuasive, unconscious form of mind control.
At this, I chuckled and scoffed. Surely, I thought, organizational “mind control” doesn’t actually exist, and even if it did, I wouldn’t be duped. Surely I’d catch on before they “mind controlled” me.
But then I recalled the stories from psychology classes in school — remembering how waiting rooms play slow-paced music to ease attendees’ impatience, and restaurants play fast-paced music to implicitly cause people to eat faster. Some logos have hidden features to cause a subconscious reaction, and grocery stores strategically place necessary items (like milk) at the back of the store to make shoppers more likely to buy something else in other aisles.
Beyond restaurants and brands, social media platforms are necessarily full of methods to subtly nudge us, the user. An app’s features, from “like” buttons to “explore” algorithms to notification frequency, are designed to keep the user on the app for as long as possible. Success is measured not in user happiness or connectivity but in time spent scrolling. If mind control is nothing more than manipulation for someone else’s benefit, then it seems deeply interwoven into nearly every app on my phone and every advertisement on TV.
Perhaps mind control is more prevalent than I had initially thought.
You might think I’m being dramatic; “mind control” is a strong term for what others might call “good marketing.” I think most people are aware of some level of manipulation in advertising and believe it’s unfortunate but tolerable. After all, it’s just a mini-manipulation here or there; one online ad surely won’t affect who I am as a person.
But the story changes when we consider the impact of hundreds of interactions with mini-manipulations, day after day, for years. Online advertising works by wearing down consumers; the more times someone views an ad for a product, the more likely they are to buy that product. Social media shapes the way we think about relationships one “like” at a time, and its prevalence makes it a powerful tool in shaping human behavior.
And it’s the design of these systems that makes them so effective. The positioning of each button on a social media page, or the color of words in an advertisement, can impact our emotions without our own recognition. A famous study showed that people mirror (and relate to) whatever emotion they see in another’s face; if they see a smiling face, they are more likely to smile and feel happy, and if they see an angry face, they’re more likely to become frustrated. The design of these systems we so commonly interact with have immense potential to shape us over time, and they’re most powerful when, just like classical conditioning, we don’t even realize they’re impacting us.
Design is astonishingly, unbelievably, unconsciously powerful.
Design has immense ramifications on our daily functioning. In a recent BBC interview, a former employee of Mozilla and Jawbone said, “Behind every screen on your phone, there are generally like literally a thousand engineers that have worked on this thing to try to make it maximally addicting.” Most of us interact with these systems dozens of times a day, and they’re designed to keep us as addicted to them as possible. We should at least be aware of that intention.
I don’t write this to add another doomsday piece to the pile of screeds against Big Tech, ringing the bell that the end of civilized discourse is near. I bring it up to drive home a more relevant point about the design of the systems we regularly interact with:
Great design isn’t just beautiful. It isn’t even the most effective or persuasive. Great design is accountable to its audience.
It demands beneficence from the designer.
If you’re still skeptical about this, I encourage you to think back to a time you were intentionally deceived. It’s uncomfortable, disconcerting, and unnerving. The initial example I shared of classical conditioning has stuck with me because in 5 minutes, somebody else changed my behavior to prove a point, and I didn’t even recognize it until I was told afterward. Even in hindsight, I’m caught between falsely convincing myself I wasn’t actually deceived and feeling frightened at how easily I was tricked in the first place.
Considering that this same unconscious manipulation is routinely attempted on anyone viewing an ad, walking through a supermarket, or browsing social media, ought to frighten us a little. It’s an intriguing but repulsive feeling, like slowing down to see a car accident on the side of the highway. I want to know how organizations are strategically capturing data about consumers, and yet I am equally afraid of how I myself am being manipulated.
I understand that these systems, for better or for worse, are here to stay. Advertising isn’t going anywhere, and it might even have some positive benefits (that’s another essay for another time). But its prevalence necessitates accountability: Designers must be aware of the power they wield and must choose to use it for good.
I’ll conclude my tirade with an important caveat: As an accounts manager at a remarkable design company, I interact with lots of designers — personally with members of our team and other partners, and professionally in viewing dozens of firms’ work and in communicating with them. With very few exceptions, the designers I work with are delightful — funny and personable, attuned with keen eyes for beautiful aesthetics, and highly devoted to building accountable design. They are not set on tethering us to screens or ads for their benefit. They care about creating beautiful work accountably.
I suspect it’s within massive corporations that people become numbers, and user experience design becomes a job to accomplish rather than a duty to perform with good intent. Our unfortunate present issue is not a result of malicious designers but of broken systems that focus too much on efficiency and too little on beneficence.
I hope to use, or at least identify, systems that are intentionally designed and maintained to solve real problems, not for the sake of continued usage. And I believe this is possible, especially as we become more aware of the ramifications of addictive, manipulative systems.
So, I encourage you to take a moment and see the design of the systems around you. Observe them for what they are and what they were designed to do, for better or for worse.
For more from Journey Group, sign up for Story Matters, our twice-monthly email celebrating storytelling in all its many forms.
