What’s Next?
Inside: how an obscure area of philosophy gives us new directions for technology and politics, and how you can help
I.
It was 2013, and Tristan Harris and I were worried about perverse incentives in tech and media. We were talking about topics that are now mainstream: fake news, clickbait, internet addiction. Both of us were tech optimists on some level — he’d had positive impact at Apple and Wikia; me with CouchSurfing and in HCI research. We both believed tech could lead to better social relations, a better economy, and better lives. But — because of these perverse incentives — things were going in the opposite direction, and we saw it would get worse.
In 2013, few recognized the problem.
We decided to split our approach: he would raise awareness of the problem. I would find ways forward for the tech industry, were it motivated to pursue them. He would be Ralph Nader, and I’d be William Whyte or Bucky Fuller.
Tristan’s work led to a TED talk, videos and articles read by millions, and extensive media coverage.
I took on two projects: First, I advocated for new measures of success in tech products, besides user engagement. This led to my talk “Is Anything Worth Maximizing?” and the Hindsight chrome extension. Secondly, I worked to re-educate designers — encouraging them to focus on how their products were shaping users’ time and social relations. I came up with a design methodology, led design workshops with Tristan, and built up a mailing list of over 1,000 designers.
II.
My projects ran into an interesting and thorny problem. Many talented and brilliant designers found the ideas exciting, but somehow intangible. Our concepts couldn’t find a place in their working minds or in their team discussions.
Only a few designers who watched the talks and attended workshops saw the product and metrics changes that were necessary. They got it. And only those teams shipped changes.
After many conversations, I traced the problem to a deep divide in how designers and engineers think about users:
Most imagine their users as defined by their immediate goals or by their apparent preferences. If their product helps the user achieve a goal (e.g., “responding to an email”) or satisfies an apparent preference (such as “seeing photos of my friends”) then — case closed — the product must be helping the user.
But ~5% of designers and engineers have a different view — they sense that users have interests or aims which go beyond their immediate goals or apparent preferences. This minority can see that a product can help with goals, or satisfy preferences, but nonetheless be a waste of a user’s time. So, a user could — despite responding to many emails and seeing many photos — eventually regret using such a product, if the product derailed a deeper concern the user has.
To bring the design and engineering community along with me, I would need to tell the 95% — oriented around immediate goals and preferences — what they were missing. But what exactly are they missing? What is more important to the user than her immediate goals and preferences?
III.
This question — it turns out — is dear to philosophers concerned with choice and agency. Economist and philosopher Amartya Sen won a Nobel Prize for pointing out how people’s interests cannot be defined solely by their present goals or preferences. [1]
Here’s my favorite example: imagine a father who raised his daughter to be obedient. But this father, much later — perhaps because of his political views about women and equality, or an appreciation of entrepreneurship — wishes that he’d encouraged his daughter to be rebellious, assertive, and free. He regrets that his prior goal for her was obedience, and he wishes he could talk to his previous self, to ask him to think about what was really important in the long run: his daughters’ obedience or her assertiveness and freedom.
There are several ways to think about what’s happening here:
- Metapreferences. One way to understand this phenomenon is that we have preferences beyond the ones we currently hold, preferences about what we would like our preferences to be. So, while the father preferred his daughters obedience while raising her, he may also have wished that he were the kind of person who would have preferred to nurture her freedom. In the 1970s, economists (including Sen) were briefly enthralled with the idea of higher-order preferences. [2]
- Multiple selves. From the 1980s until the early 2000s, another approach came into vogue. Perhaps the girl’s father had two selves: an instinctual self with one set of preferences, and a reasoning self with another set. Economists and psychologists like Daniel Kahneman and Tyler Cowen advanced multiple-selves or multiple-sets-of-preferences models. [3]
- A process of reflection / discovery / clarification. But by the late 2000s, those concerned with this problem had settled on a new view: the father had simply been wrong about what he wanted. When we choose, we do so based on a rough guess of what our true interests are, and we are always looking to improve that guess by finding better goals and values. [4]
In the first two theories, the father is conflicted in a deep way. But this doesn’t explain why the father would like to talk to his former self. If he did, he would still have multiple selves or preference sets.
The third theory does explain why talking would help. It says that change comes through participation in a process — of reflection and discovery and clarification — leading to a better understanding of which goals and preferences we’ll stick with and which we’ll discard. Talking with one’s former self could accelerate that process.
So there’s our answer: What is even more important to a person than their current goals or preferences? The process of refining, discovering, and clarifying those goals and preferences.
IV.
I believe this idea amounts to a new and transformative view of human nature. If enough people start to see themselves in this way — as participants in a process that updates their goals and preferences — it will transform society.
Society moves forward when the design principles for human systems change. When a new principle (like “freedom” or “fairness” or “meritocracy” or “structural oppression”) appears and becomes widely recognized, people redesign everything (schools, local businesses, etc) in accordance with it.
Today, many principles vie for this status: being “distributed”, “peer to peer”, getting “skin in the game”, having a “flat hierarchy”, being “commons-based”, “sustainable”, “zero-knowledge” and so on. But none of these have what it takes to transform society.
I conducted a study of which principles worked in the past. Those which transformed society were deeply connected to our human experience, led to new forms of intimacy between human beings, and ignited the public imagination with clear visions of change.
The process-oriented view meets these criteria. What society would help people determine their most meaningful goals and preferences? This question sparks the imagination. And the related design principles — exposure to the consequences of one’s actions, discretion over the manner in which one works, and fostering capacity to handle hard truths together — suggest immediate changes to institutions and new forms of intimacy.
This vision — and these principles — are inspiring and practical. They compare well against the drab future-visions we’ve been fed: visions like Fully Automated Luxury Communism, a Return to Liberal Ideals, Economic Nationalism, Green Technocratic Management, and Blockchain Crypto-collectivism.
V.
So, how do we give this new vision of human nature its due?
I’ve worked for the last 18 months to package it up into a training program to run at interested companies and organizations around the world, with a handbook of activities and games, and a series of essays and videos.
Right now, I’m gathering local organizers and hosts in cities and at tech campuses around the world who are interested to play some of these games, gather some of their coworkers, and explore a new way to think about technology, society, and design.
Notes
- Sen’s influential work on the topic was Sen 1977.
- See Hirschman 1985 for a tour of the metapreferences literature.
- Kahneman 2002 and Cowen 1991 are representative of this approach.
- I refer to a particular process-based view, combining three ideas: first, that we don’t tend to act when we have only desires but no reasons; second, that to have a good reason is to have undergone a process of reflection which has succeeded; third, that such success is an endorsement based on our “identity”: our sense of who we believe we are and who we believe in being. By 2002, Sen had switched to this view, but it is already present in Velleman 1985, Korsgaard 1992, and Quinn 1993.
