Dark Wisdom: How Design Manipulates The Way We Think, Feel and Behave
This essay elaborates on the TedxDelft performance I did in 2013 called “Why design eats advertising for breakfast”. During the past two years I got more and more upset with how powerful forces shape our emotions, thoughts and behaviors. Governments, corporations and social movements apply the dark principles of behavioral design to exert power over nearly every aspect of our daily life. In this essay I want to explain how evil design get us to obey, how bad design triggers unwanted behavior and how good design can liberate us from evil and bad design.
Design eats advertising for breakfast
I’ve always been fascinated by the subject of influence. My first real plan in life was to become a therapist, but I ended up owning an advertising agency. Basically, my job is to seduce and persuade people for brands and products (the evil part) or to get them to donate for charity or to stop wasting energy (the good part). Now what really strikes me about the advertising business is that our most import instrument of influence — creative communications — is actually a pretty lame one. We keep clinging onto the naif belief that — as long as we keep communicating in an original and persuasive way, people will eventually change their behavior. The fact that advertising is becoming one of the most polluting forces today, probably is the best proof, both of this stubborn belief and of the failure of it.
At the same time, when you look at how governments, corporations and social movements influence us, you can see forces at work that shape our emotions, thoughts, attitudes and behavior in way more powerful ways than persuasive communication could ever achieve. I want to argue that nearly all our thoughts, feelings and our behaviors are unconsciously shaped by behavioral designers. I see behavioral design as a dark science or wisdom, that consists of rules, principles and tactics that ultimately aim at getting us to act. I think there are only three kind of behavioral designers: evil designers, bad designers and good designers.
- Evil designers: People and institutions that deliberately use design principles to force us to comply to their will.
- Bad designers: The big group of people, institutions and belief systems that unintentionally trigger the exact opposite behavior they intend to shape.
- Good designers: People that use design principles to liberate us from the influence of evil and bad design
I think bad design is far worse than evil design, because it always takes a while before we learn to notice how the unwanted behavior that follows from a badly designed system is actually triggered by the system. Let me start by exploring the nature of evil design.
How Evil Design Forces Us To Obey
When the Nazi’s gave green light for the Holocaust at the Wannsee Conference in 1942, they put the overzealous officer Adolf Eichmann in charge. While planning the deportation, he quickly discovered after a couple of trials that the best way to get his victims to get on the train to the extermination camps, was a simply redesign of the check-in procedure. When you invite people to show up for registration and ask them to take one bag with their most important belongings, and you carefully register the bag, you basically design a procedure that gives hope for a happy ending. You make them believe that you will reunite them with their belongings. Laurent Binet explains in his brilliant book HHHH why the bizarre high rate of compliance and the lack of resistance to the call for deportation.
Another example of evil design that filled me with absolute shivers is the concentration camp of Sachsenhausen near Berlin. When I visited this camp a few years ago one observation hit me in the face: Next to the entrance of the camp was an SS training school. The sickness of this design was that it actually designed horrendous behavior: Young SS’ers were triggered to show their courage and creative cruelty to their peers and superiors by picking out random camp prisoners and play sadistic games with them. Some evil designer must have figured this out. The behavior that follows from it is fairly easy to predict. There’s a lot to be learned from these dark episodes of our history, by looking at evil behavior, not as a trait, but as a fairly predictable consequence of the design that triggered it.
Evil design is deeply rooted in every aspect of our daily lives. The reason why our smartphones are so addictive is because the brightest social engineers are constantly working on getting us addicted. Great software is all about designing habits, as Nir Eyal brilliantly demonstrates in his book Hooked. The apps that hook us (Facebook, Twitter, WhatsApp, etc…) all have on thing in common: Social engineers came up with subtle ways to constantly lure their users back into using the app with little bleeps, dynamic avatars, status notifications and a never ending stream of e-mails. Social Engineering is the new discipline that basically aims at addicting us to the apps and interfaces around us. And the amount of money and talent these billion dollar companies throw at getting this right, is way bigger than the money and talent that goes to designing healthier behavior or social programs.
In a recent article on the big success of the collaboration app Slack, the lead designer writes about its secret sauce. It’s interesting to see how the makers have thought about every little detail in order to get users to use, reuse and evangelize the app:
Like a well-built home, great software focuses on giving its users hundreds of small, satisfying interactions. […] In Slack, every piece of copy is seen as an opportunity to be playful. Where competitor might just have a loading spinner, Slack has funny quotes like, “Need to whip up a dessert in a hurry? Dump a bag of oreos on the floor and eat the oreos off the floor like an animal.” A strange little injection of fun into an otherwise boring day. Slack acts like your wise-cracking robot sidekick, instead of the boring enterprise chat tool it would otherwise be.
Once one starts looking at persuasive communication from a design perspective, one could easily see that there’s social engineers at work in shaping the desired outcome of corporate and political campaigning. Every populist knows that in order to win the hearts, minds and — eventually — the votes of the masses, one simply needs to have them to look at things through their frame. Fox News can be seen as nothing more than a machine that keeps on reframing everything into a white Republican paternalistic narrative. Protestors are terrorists, Diplomacy is for pussies, Corporations are heroes, the Unemployed are Lazy, Punishment is Good, etc… The Russians learn to see the Ukrainians as terrorists. The IS soldiers learn to see the West as the imperialists that want to destroy Islam. Our European governments want us to look at the Greek as being stubborn and lazy,… the list is endless. The latest elections in which the Republicans won the Senate in November 2014 can be seen as an incredibly disciplined and orchestrated campaign to win at any cost, as pointed out by The New York Times in this OpEd “Negativity wins the Senate”. The architect behind this coordinated storyline design is a spindoctor called Frank Lutz.
Republicans would like the country to believe that they took control of the Senate on Tuesday by advocating a strong, appealing agenda of job creation, tax reform and spending cuts. But, in reality, they did nothing of the sort. Even the voters who supported Republican candidates would have a hard time explaining what their choices are going to do. That’s because virtually every Republican candidate campaigned on only one thing: what they called the failure of President Obama. In speech after speech, ad after ad, they relentlessly linked their Democratic opponent to the president and vowed that they would put an end to everything they say the public hates about his administration.
How Bad Design Triggers Perverse Behavior
The Dutch anthropologist and journalist Joris Luyendijk kept a blog and interviewed finance people in the City of London for more than two years for the Guardian. He wanted to find out how on earth this financial crisis could have happened. His conclusions (to be published in the upcoming book “How Can You Live with Yourself”) are terrifying. What he discovered was first of all that bankers are not evil as such. If it was only a matter of locking up the bad guys. What he discovered was that the financial world is a sick system that triggers risky, competitive and ultimately destructive behavior. When you design a system in such a way that the awards are for the bank but the losses for the society, you basically provide a license for big gambling. This simple design triggered all kinds of perverse triggers, from the excessive bonusses for the winners, to the evil HR-system “Rank and Yank” that Enron introduced, in which the bottom-performing 10-20% of all employees gets sacked every year. This of course triggered employees to engage in all kinds of manipulative bookkeeping in order to game the Yank and Rank system, which eventually led to one of the biggest bankruptcies in history. One of the Investment Bankers that Joris Luyendijk interviewed perfectly summarizes how the design of the system works.
“Investment banking is a trap, a game and an addiction. The reward is big, but uncertain, which makes it exciting and keeps you coming back for more. Once the money starts flowing it’s very, very hard to take yourself away from it. Doing a deal is like scoring a goal, or maybe for journalists, getting a scoop. The game element is in the rivalry with other teams, winning the mandate, legging over the competition … Also the emptiness that comes with addiction.”
It’s not that difficult to understand that Luyendijk is very pessimistic about the future. The recent inflow of cheap money back into the financial system is basically fueling the old game back to unmatched levels, something Nobel Prize Laureate Joseph Stiglitz has been warning for since the early days of the crisis.
The problem is that it’s not just a design mistake in the fabric of our financial institutions. Bad design and perverse triggers are everywhere. Every belief system contains its own sets of triggers. Communism started as a good idea, but eventually designed the biggest and deadliest oppression machinery in history in order to get people to obey the system. Liberalism once took off as a belief system that aimed at liberating the individual from the oppression of the big centralized ideologies like Catholicism, Communism and Nationalism. And for a brief amount of time it provided some beneficiary effects. It made Francis Fukuyama write the famous essay “The End of History”, in which he proclaimed that history is now completed with the free individual as the final stage of historic progression.
Twenty years later we wake up in a world in which progress and value is only measured in economic terms. In which success means money “(If you’re so smart, how come you’re not rich”). In which not being successful means being a loser. In which managers reorganize every aspect of our lifes to make it more profitable. And in which the biggest surveillance system ever imagined is tracking our actions, and analyzing our behaviors for the sake of liberating us from any possible risk. To think of Liberalism as the end of history turned out to be a dangerously naive idea.
Every ideology first liberates us from the excesses from the previous ideology, but then inevitably installs its owns set of perverse triggers. This is what’s known as Shirky’s Principle:
“Institutions will try to preserve the problem to which they are the solution.”
The Dutch novelist Tommy Wieringa recently provided a brilliant insight in the mind of government officials in a lecture called “The State as Pimp”.
I translated it from the original Dutch transcript:
The Utopian in its full glory is nothing more than a dry bureaucrat who arranges both the landscape and the soul according to his preference. His ‘Fantasy Product’ demonstrates a remarkable lack of imagination as well as a surprising — if not alarming- limited arsenal of themes .. . […]. In his mind the world looks like a chessboard, populated by pawns that move when and where he wants them to move. Nobody can escape this neurotic paternalism, which — according to Kant — is the worst form of despotism one can possibly imagine.
In order to organize our liberal utopia, we have started to measure everything. And once you start to measure everything, you start to organize the world in order to improve your metrics. And once you start thinking like that, you start to see people as the annoying things that fuck up your KPI’s. And before you know it, the manager thinks he’s more important than the working force or production force he’s managing. A miller needs to pay 400 EUR every year for an authorized mice exterminator, because the government doesn’t trust the miller to keep his mill clean by himself. The hospital shuts down its psychiatric beds because these beds bring down the hospital performance score. The performance management system punishes the hospital for longer stays. A University decided to shut down most of its human science departments because they bring down the economic output metric of the University. This phenomenon is called Goodwin’s Law: every metric that becomes a KPI stops being a metric.
Let me repeat: bad design is far worse than evil design, because it always takes a while before we learn to notice how the unwanted behavior that follows from a badly designed system is actually triggered by the system.
Good Design: How To Use Dark Wisdom For Good?
The biggest problem with this dark wisdom is that those with a desire for power and dominance are more keen to embrace its principles than those who want to do good. Do-gooders on the left, always tend to convince us of our moral duty. They believe we will eventually do the right thing if only we could see things the way they see it and if only we could realize that it’s the right thing to do. I saw a heartwarming video the other day that went viral on social media.
A young intelligent black activist woman, called Chloe Valdary held a very eloquent plead against compassion by the progressive elite for the Baltimore rioters. She argued that showing understanding for the rioters and seeing it as an expression of the feeling of powerlessness of the black community, simply is racism is disguise. It’s like treating black citizens as helpless children who can’t express themselves otherwise. It’s very tempting to agree with this argument, as 50.000 people on Facebook did. The problem is: It’s not that it’s wrong, but it doesn’t change a thing. Richard Rorty, one of the biggest philosophers of the 20th Century once famously said:
“We resent the idea that we shall have to wait for the strong to turn their piggy little eyes to the suffering of the weak, slowly open their dried-up little hearts. We desperately hope there is something stronger and more powerful that will hurt the strong if they do not do these things.”
Rorty despised utopian thinking and thought of philosophy as nothing more than a nice literary genre. Fun to read and fun to discuss, but holding no relation with reality. If you want to shape the world into a better place, than the only thing you can aim for is to convince the strong that not being evil is in their own self-interest. He pleaded for a far more pragmatist approach to changing the world. Instead of thinking about what people should do, think about a way to get them to act.
If you want people to donate to charity, the most effective strategy by far is still to make it really difficult to say no to the cute girl who just rang your doorbell. If you want to prevent kids from smoking: turn it into an addiction they simply can’t afford. If you want bankers to stop gambling, make them personally accountable for the losses. If you want kids to get passionate about learning: turn geekness into something incredibly cool. If you want your cops to stop using excessive violence, stop providing them with military materials in the first place. If you want to prevent Western muslims from radicalizing, start offering them opportunities to earn more respect in our society. If you want your employee to work hard, simply walk around every day and ask them what they are doing and if you can help. If you want people to recycle: make it more difficult for them not to recycle by offering transparant bags that look really filthy when you throw organic waste in it. If you want people to get more energy efficient, simply show them how much more they pay for their bill than their neighbors.
The list of behavioral interventions is endless. All we need to do is convince the strong to implement them for their own good. :-)
Tom De Bruyne
Amsterdam, May 2nd 2015
PS: This essay is a rework of the TedxDelft talk I gave in 2013 called “Why designers eats advertising people for breakfast”. I’m in the process of writing a book on this subject. If you want me to keep you posted, please leave your name and e-mail address here.
