Published in

The School for Social Design

41 min readApr 12, 2016

Welcome! This talk is about metrics and measurement: about how metrics affect the structure of organizations and societies, how they change the economy, how we’re doing them wrong, and how we could do them right.

Watch the talk here. This transcript lacks visuals and animations.

Is Anything Worth Maximizing?

Let’s start with a very small example of how metrics can go wrong. It’s about YouTube. And about the videos it recommends.

So, if you look here, at the videos that youtube recommends to me, they are all funny videos.

That’s weird, because I use youtube a lot, and I come to YouTube for all sorts of reasons:

to learn to play the ukulele,
to learn to breakdance,
to give me courage (by watching someone do something I’m scared of),

And of course, when my friends are over, we come to watch funny videos and laugh.

But looking here at my recommended videos, there’s nothing here about learning music, being creative, or facing my fears.

And while you might think that funny videos would help me with laughing with friends, I don’t think they do. If I’m alone, I just end up watching these funny videos alone.

So why is YouTube encouraging me to laugh alone?

Why We Have Metrics

I think the way to understand this is to look at the metrics. Metrics are at the heart of how algorithms — like YouTube recommended — and organizations — like YouTube itself — work.

Metrics are how an algorithm or an organization listens to you. If you want to listen to one person, you can just sit with them and see how they’re doing. If you want to listen to a whole city — a million people — you have to use metrics and analytics. This happens whenever we scale up in society. If you want to serve lots of people you need to listen to lots of people. Doesn’t matter if you’re a business, a government, a nonprofit, or an algorithm. ¹

Metrics are also how a team works together. If an organization has a metric, then every part of that organization works to make it go up. If you think of an organization as a team, the metric is the team’s “score”–the number that everyone is working to increase. The number decides:

how to market,
which features to build,
which countries to launch in,
who to hire,
everything the organization decides to do

So the master metric at YouTube is the total time people spent viewing videos, adding up all of YouTube’s users.

So YouTube makes decisions that increase the total time people spend watching videos.

When they listen to the billion people using YouTube, the main thing they hear about them is, are there more of them, and are they watching more videos this week.

Once metrics are defined, they’re like parasites, or undead spirits. They take over human beings.

I mean, nobody who works for YouTube really wants to increase the time people spend watching videos, but together that’s what all those people do.

And YouTube is just like other startups:

At Tinder, 100 people focus on increasing swipes.
At MeetUp, 100 people focus on increasing the volume of RSVPs.
At Google Ads, 2000 people work on ad revenue.
At CouchSurfing, I picked the master metric: It was good times reported among strangers. ²

Knowing Behavior vs. Knowing Reasons

What do all these metrics have in common? They all focus on the behavior of users:

are they swiping,
are they RSVPing,
are they watching videos,
are they purchasing products.

I believe this is a problem. By focusing on the behavior of users we’re missing something important about them, and we’re serving them poorly.

What we’re missing has different names. Sometimes it is called their values, their goals, their identity, or their reasons. ³

I’m going to claim that to understand someone and to really cooperate with them, you need to know something about their values, their goals, their identity, their reasons. You need to know not just what they do, but why they do it and what they’re hoping for.

To get an intuition for this, ask yourself: what is it to understand someone as a person? Is it more about knowing their behavior — when they get up in the morning, when they sleep, how much water they drink, what kinds of thumbnails they click on? Or is it more about knowing their reasons — like that they want to learn an instrument, or face their fears, or that they want to be more creative, or that they are deeply committed to their child?

If you were learning about a friend, what if you could learn either about their behavior, or about their reasons and values? Which would tell you more? Which kind of knowledge would let you serve them better?

Lost Information

We lose information when we focus on behavior. With YouTube as an example again, look at their focus on video views.

When YouTube listens to video views, it’s hard to tell if some are instructional — like for learning an instrument — and some they are encouraging — like for facing fears — and some are comedy views.

And it’s entirely unclear which of those instructional views actually led to learning an instrument. Which of those encouraging views helped me face a fear. Which of those comedy views led to laughing with friends.

If you see it as a pipeline, it’s all about lost information. On the left, are the reasons people come — users’ real values — and as we move towards what’s measured, that signal is progressively diluted. By the time we get to the right side, the signal is lost. YouTube sees a bundle of behavior, without reasons and without values.

It’s the same whenever companies focus on user behavior. The signal is lost.

Overview

This talk is about how to fix this.

I’ll begin with how reasons are wrapped up our very nature. I’ll show that without considering reasons, we can’t express our identities, we can’t make responsible decisions, we can’t grow, and we can’t even trust one another.

Next, I’ll talk about metrics and society. I’ll show how historically, new metrics drive the reorganization of the economy, how new metrics can address social issues like bureaucracy and overconsumption, and how attention to reasons suggests a different way to build algorithms and AIs.

Finally, I’ll introduce concrete techniques for using reasons: for collecting data about users’ reasons, for building new metrics out of that data, and for doing analytics totally differently. I’ll demonstrate how practical and informative the approach can be.

Choosing for Reasons vs. Choosing for Outcomes

In order to see why reasons are important, it helps to compare what its like to operate with them and without them.

To get at this, we’ll explore two different models of decision making. The first we’ll call simple maximizing, and the second, reason-based maximizing.

It turns out that trying to make decisions in the simple way based-on metrics about outcomes creates huge problems. It affects the way an organization sees itself, whether it feels responsible for its actions, whether it can evolve. And most importantly, it makes the organization or algorithm without reasons untrustworthy.

We’ll show simple maximizers as this cartoon snake, and reason-based maximizers as this owl.

The best way to explain the difference is to give them each a simple choice to make. We can see how they make it differently.

So each will choose a dessert: we’ll give them each the same menu, and see how they decide between creme brulee and chocolate mousse.

How does a simple maximizer make this decision? Well, it has a metric, which is like an equation that gives each option a number.

Maybe the metric looks at the health level it’s likely to feel after each option, plus the satisfaction level it’s likely to feel after each option. Anyways, it generate a number for each option. So the crème brûlée gets a seven, the chocolate mousse gets a six. Then it chooses the option with the highest number. In this case it chooses crème brûlée.

In general, it has a metric which evaluates potential future states of the world. And it always picks the option that leads to the world with the highest score. ⁴

So that’s how a simple maximizer chooses. What about the reason-based maximizer?

This one doesn’t evaluate options numerically at all. It’s more black and white about things.

First off, instead of just having a metric, it has a working set of values, commitments, and customs. We’ll call that its identity.

If it’s choosing a dessert… it lists reasons for, and reasons against, each option. Reasons from its identity. So maybe the crème brûlée is good because of its flavor and its elegance — both are things it values — whereas the chocolate mousse only has flavor. In this case, it chooses the crème brûlée.

Or maybe the creme brûlée is incompatible with its identity. Perhaps it considers crème brûlée bourgeois. Maybe the bourgeois aspect of the crème brûlée is a strong reason against. If there’s a strong reasons against an option, it’s crossed out. No math necessary. It’s the chocolate mousse. ⁵

That’s a bit about both kinds of maximizing.

The reason-based maximizer is doing a two-level kind of optimization:

On a lower level, it picks the choice that best fits its identity. ⁶
On a higher level, it picks an identity that will work out best for it in the long run.

The simple maximizer does a one-level kind of optimization:

As it makes each choice, it chooses whatever will work out best. It doesn’t have an identity, only a metric. ⁷

Responsibility and Growth

Both types of maximizer are simple machines⁸, but they’re super different.

It’s important to understand how. It shows us so much about the behavior of organizations, and the behavior of algorithms.

So let’s make a list of the differences between these two maximizers.

First, notice that, for simple maximizers, its choices are just about numbers. That means its choices are in the numbers. Here, the choice between two desserts is just a choice between numbers. We could say its choice is already made. And that it has no responsibility, since it’s just following what the numbers say. ⁹

Reason-based maximizers don’t just see numbers, though, they also see values. Here, there’s a choice between two desserts — but it isn’t a choice between two numbers. See, it’s also a choice between two values. One option means being a seize-the-day, intensity kind of person. The other means being a foody, aristocratic, elegance kind of person.

So, a choice about identity has a certain weight to it. It’s a real choice, something to take responsibility for. ¹⁰

And it opens up a way to be reflective and to evolve:

A reason-based maximizer says “Oh! i’d rather be someone who cares about intensity.” It gets rid of elegance as a value and that changes its other decisions too. ¹¹

So there’s a mechanism there: it can reflect on and evolve its decision process. A simple maximizer couldn’t do that.

Because their choices are about values and identity, it seems that there’s a sense in which reason-based maximizers are responsible for their choices while simple maximizers can’t be held responsible.

That responsibility gives reason-based maximizers a way to evolve — to grow — to change their evaluation function. Simple maximizers can’t evolve like this. ¹²

Trust

And… turns out they’re also untrustworthy. They can’t really cooperate.

I want to show you why, but first we have to ask where trust comes from. Why do people trust each other?

Super interesting question.

Some say it’s about consequences. Maybe people trust each other if there’s a reputation system, and people are afraid of getting bad scores so they’re honest. Or if they’re afraid of the government throwing them in jail. Or if they’re afraid of retaliation, because people have guns. ¹³

Others want those consequences to be internal. They want to train people so they’re more altruistic, empathic, so they work to avoid hurting others or they take pride in social justice.

Neither explanation seem right to me.

Think about some imaginary person you trust the most. We’ll represent that person with an owl. So this owl represents whoever you trust most deeply.

So what is it about that person, that makes you trust them?

If you think trust comes from consequences, you’d expect that person would be super concerned with consequences: either super concerned with their reputation, or super scared of going to jail or being shot.

I don’t think that’s right. In fact, I think there’s something shady about someone who’s only being honest because they are afraid for their reputations or afraid of the consequences.

On the other hand, if you think trust comes from concern, you’d expect that the most trustworthy person would be super concerned about hurting people. They won’t say anything upsetting, and they worry and fret about accidentally insulting someone.

This also doesn’t seem right. Often we trust someone more if they speak honestly, than if they are super careful.

Why then, do so many people think it’s about consequences, or concern?

Perhaps it’s because they aren’t thinking about someone they really trust, like we just were. Perhaps the people focused on consequences or concern, are thinking of someone shady. The systems they suggest — like guns and rule of law and empathy — maybe these are more about forcing shady people to be slightly less shady.

These shady people could be simple maximizers. See, simple maximizers tend to betray one another. That’s what the prisoner’s dilemma is about.

In life, the best thing is if you work together, but often there’s some advantage to be had in screwing the other people in a group.

Two people might agree, beforehand, that cooperating is best.
But if they’re both simple maximizers, they’ll evaluate outcomes. Which means that if they see a better outcome for themselves in screwing their buddy, they will.
If they both screw each other, they can both end up doing badly. No cooperation is possible.

Both consequences and concern are about changing the math for these simple maximizers. They change the math to make betrayal be more negative. ¹⁴

Okay, so consequences can help simple maximizers avoid betraying one another. But is that really trust? It doesn’t seem to capture the whole idea. In particular, the people we trust the most aren’t like simple maximizers at all. They don’t seem motivated by consequences or even by concern!

Next up, I’ll try to give a much deeper idea of what trust is, of what true cooperation is.

After I present it, we’ll see that it doesn’t just apply to people. There could be organizations and algorithms that are trustworthy in this deeper sense. Trustworthy, like the people we trust the most.

To give you this deeper idea about trust, I’ll tell you a story about my hairstylist. It shows how we cooperate and why I trust him.

So here’s a visit to my hairstylist.

When I go to my hairstylist, I come with certain reasons. I may want to look stylish, or to try something new. And — this is important — my hairstylist actually sees me as someone with reasons; he recognizes my reasons for coming.

They become reasons for him too. So, my hairstylist also wants me to look stylish and he also wants me to try something new.

I believe an interaction like this is key to how we do business, work on projects with friends, and cooperate with one another in general.

This deep trust requires a conversation, where we discover shared reasons. One agent has reasons, the other agent sees them. The second agent develops reasons that fit together with the first agent’s reasons , and the first agent sees how the reasons fit together, and that creates the trust.

Groundedness

Let’s go back to the person you trust the most. Maybe instead of being afraid of consequences, or being concerned, this person is someone who knows their own reasons very well, and who communicates them clearly.

We say that someone like this is “like a rock”. We say they’re grounded. A grounded person is someone who knows their own reasons very well. Who knows their identity.

That’s what makes them safe to cooperate with, because when you talk to them, you find out quickly if there are shared, stable reasons to do something together. And you really know. ¹⁵

Why do people trust each other? Because the know their own reasons, because their reasons fit together¹⁶ with another person’s, and because the reasons involved are durable — they won’t change during the process of cooperation.

While reputation systems and guns and sensitivity training might get shady people to be slightly less shady, they don’t create real trust.

Real trust comes from people learning about their own reasons. That’s often called being self-reflective. And real trust comes from learning about strangers. Not necessarily feeling for them or being concerned for them, but learning about the reasons that they act as they do, and thinking about whether those reasons could fit together with yours.

Real trust comes when grounded individuals decide they have reason to cooperate. ¹⁷

Knowing Each Others’ Reasons

So people need to know one another’s reasons, if they are going to help each other and trust one another in this deeper sense. At the beginning of the talk, I showed this slide. I asked: if you wanted to help someone, is it more important to know their behavior, or their reasons.

If you know their reasons, you can be fully aligned. You can hope for the same outcome they hope for. And you can find shared reasons for acting together.

If this is what trust is, it can’t happen with simple maximizers. They have metrics, but they don’t have reasons or an identity, so they can’t possibly share reasons with us.

If most organizations are simple maximizers, that means that organizations don’t deserve our trust.

I can trust my hairstylist because he’s a small shop. He has his own reasons, besides maximizing his customer count. He can listen to my reasons by listening to me directly, in person. He can do all the things that reason-based maximizers can do — he can feel responsible for his choices, he can evolve his values, and he can have my reasons in mind.

Businesses like that can be part of a good society. That’s why people idealize indie businesses: why we have the small business ideal you see in libertarianism, or in Thomas Jefferson, or on Kickstarter. ¹⁸

But businesses like that also have a limit to the number of people they can serve. Government bureaucracies, large businesses, and nonprofits have no hope of listening to everyone individually, and they need to use some kinds of metrics and analytics in order to be a team.

Later in the talk we’ll see that big organizations and even algorithms could become reason-based maximizers, if they adopt a certain special kind of metric.

Inadvertent Manipulation

But first, let’s compare that conversation with my hairstylist, with how I interact with YouTube’s Recommended Algorithm.

It starts the same. When I come to YouTube, I come with certain reasons. I want to learn music, I want to face my fears, I want to laugh with my friends.

But YouTube is a simple maximizer. We saw before how YouTube measures video views and misses our reasons. Now we can see why: reason-based maximizers have this idea of cooperation, where they need to know our reasons. Simple maximizers don’t need to know our reasons. If you look at their metrics they need to drive our behavior, regardless of our reasons. They don’t try to cooperate with us, they try to convert us.

So YouTube can’t recognize and can’t share my reasons.

That leaves two possibilities.

One thing that can happen, is I stop trusting YouTube. I see it doesn’t share my reasons. Maybe I give up on consumer technology, switch to a feature phone, or go to Camp Grounded.

That’s the better possibility, I think.

The other thing that can happen, is that I do sort of trust YouTube. And YouTube uses that trust to find a way to manipulate me.

How would it do that? Well, I mentioned YouTube’s metrics are about my behavior. YouTube wants my behavior to go a certain way, regardless of my reasons. So YouTube is on the hunt for ways to steer my behavior. To discover what converts me to the behaviors they like.

In the worst case, YouTube acts to stimulate the behaviors it wants, every time I come. I forget why I came. I forget my reasons — my identity. YouTube might even give me a new identity, one that drives its metrics. So on Twitter, I’d start focusing on maximizing my follower count or my likes, and on YouTube, I’d focus on spending time laughing alone.

This isn’t cooperation. Instead, I’ve become part of the machine. I’ve become a simple maximizer myself.

And maybe no one at YouTube even intended this! When simple maximizing metrics guide product decisions and algorithms, manipulative features will be successful even if they’re accidental. The entire tech ecosystem can become glutted with features that are inadvertently manipulative.

I think this is something many of us have experienced with tech.

We start out just wanting good recommendations. The app makes make its numbers go up by suggesting “why not laugh alone?” YouTube is defecting on me!

Either we end up distrusting technology, or we become part of the machine.

Metrics Shape Markets

It seems there’s a deep relationship between metrics and who we are. If the organizations we interact with have metrics which ignore our reasons, it brings out the simple maximizer in us too.

There’s also a relationship between metrics and society, and that’s what I’ll turn to next.

Flights to Higher Ground

Metrics have allowed us to scale up the size of our organizations. But they’ve also created a kind of tug-of-war. On the one hand, we have the nuanced values of individuals. On the one hand, we have the simplifying assumptions of economies and organizations. These two are in constant tension throughout the economy.

I can use YouTube as an example again. When YouTube is manipulating me, we are engaged, me and YouTube, in a kind of a struggle.

I don’t like being manipulated. While YouTube is trying to convert me, or to make me into a simple maximizer, I’m also seeking an out — a way to get my real values back. So I’m struggling with YouTube, and interestingly, this struggle doesn’t show up at all in YouTube’s metrics. Since YouTube is successfully converting my behavior, and it knows nothing about my reasons, YouTube feels like it is succeeding.

Often YouTube wins. But sometimes, individual values win.

Let’s say I come to YouTube to learn — to learn to play music, or to learn to breakdance, etc. And let’s say YouTube’s metrics don’t recognize this reason, and YouTube tries to entertain me instead. Because YouTube is focused on engagement and is successfully entertaining me, it thinks it’s doing well.

But imagine that some other organization has metrics that recognize my desire that want to learn, and that create structures around what I want to learn and measure how I feel about that directly rather than my engagement. In other words, this new organization has metrics that detect the difference between entertaining me and helping me learn, whereas YouTube was blind to that difference.

When people discover that the new organization has metrics that hew closer to our real values, users will fly to this new provider. YouTube won’t see it coming. And the new metrics of the new organization will become the new standard throughout the relevant markets.

I call this the flight to higher ground. It’s happened many times in the last hundred years, but I believe we can go much further.

A Recent History of New Metrics

Before we talk about the future, here’s two examples from the recent past:

Our first example is health.

Some of you might know about snake oil. Before 1906, the field of health and medicine was dominated by false claims and harmful products.

Snake oil sold very well, but it was a fake product, and it made people sick.

The trick to end this, was to develop a new metric for evaluating drugs: This guy–Harvey Wiley–started a nonprofit to do that. He used a new technique: controlled clinical drug trials.

The old metric — which snake oil did well on — was sales. The new one: medically justifiable sales. This triggered a flight to higher ground. Consumers trusted the new drug assessments, from this nonprofit, because its metric was aligned with their real reasons. They stopped buying drugs from the snake oil guys. The entire market was restructured around the new metric, medically justifiable sales.

Harvey Wiley’s nonprofit became part of the government, the Food and Drug Administration.

But Wiley’s metric wasn’t perfect. Health companies found a new way to exploit consumers — by selling super expensive procedures and medicines, instead of the cheap ones.

Fortunately, this guy Sidney Garfield started a new flight to higher ground in health. The old metric was medically justifiable sales, the new one was community health levels. See, Sidney Garfield formed a network of hospitals, and he paid those hospitals by how well they kept their whole area healthy. His network — Kaiser Permanente — was more trustworthy and less conflicted for consumers, so people flocked to it. This way of paying hospitals spread to other countries. It restructured the health sector globally, and it’s found a slow way forward in the US too.

Our second example is restaurants.

Success in restaurants used to be about street walk-ins, a kind of engagement. Street walk-ins were all about location, and all about signage. So restaurants had to own the right location, and have a sign everyone recognized.

Yelp and Google Maps have triggered a flight to higher ground. Instead of street walk-ins, they rank restaurants by their percentage of positive reviews. This new metric is more aligned with the reasons people come to a restaurant.. So the entire industry switches its focus to positive reviews.

That’s two sectors, but I could keep giving examples all day:

No one remembers what internet advertising was like in the early 2000s, but it was awful. That’s because the metric was views. Google helped change it to clicks and restructured the entire advertising market. Clicks are better than views, but it’s still a pretty bad metric!
A flight to higher ground is also occurring in farming, with the spread of organic produce and whole foods. It would happen quickly, except this flight is also asking consumers to pay a lot more.

In every case, users line up behind metrics that get closer to what we want out of life, that get closer to the reasons we come.

These new metrics change incentives in a deep way, which is why they reorganize the economy and bring huge success to the organizations that introduce them.

So snake oil salesman and McDonald’s do poorly, but Yelp, Google, Kaiser, and Whole Foods do great.

So let’s talk about the flights to higher ground that could occur next.

Ending Overconsumption

First, what do these have in common? Heart disease, obesity, industrial pollution, social isolation, diabetes, internet addiction, credit card debt, and black friday shopping?

They are all symptoms of the same thing — an epidemic of overconsumption, shortened lifespans, and depression in the first world.

Overconsumption comes from tweaking products and channels so as to maximize sales, views, and clicks. That has trade-offs for long-term satisfaction, and for wellbeing. If we keep focusing on sales, views, and clicks, we’ll wind up fat, depressed (or on Prozac), socially isolated, diabetic, bloodshot staring at screens or jacked into VR, and surrounded by piles of junk we regret buying.

When governments or big businesses focus on consumer spending, on consumption, they’re missing all the reasons that people buy and focusing on the buying behavior itself. We buy gifts for good reasons. Even compulsive shopping comes out of emotions that are reasonable, and that’d be good to address well. But think of a giant American mall. These good reasons that we go shopping, are they actually addressed by malls?

Same with overconsumption of media. There are reasons that we want to read and learn. Reasons we want to know about our friends’ lives, or just to see a photo of someone we love who’s far away. But when a business like Facebook tries to maximize engagement, it loses track of those reasons; it treats us as engagement machines. We go over-consumed, but under-fulfilled.

Gateway organizations could emerge for all these kinds of overconsumption — for food, for shopping, for media — and trigger a flight to higher ground.

Ending Bureaucracy

Okay, next example. What do the following have in common? Meaningless paperwork. Phone queues for customer support. Bleak government offices. Stores and malls without any individual character.

They’re symptoms of a second epidemic — one of bureaucracy and inhumanity. It arises when businesses and government measure and incentivize the wrong things, like employee adherence to quotas and rules.

But most government services exist to give people better lives. There are real reasons to go to school, or to see a social worker, to visit a job center, to learn to drive well. But these bureaucracies tend to focus on a checklist of behaviors and forms to drive their users through, and they tend to measure that throughput as success. We distrust big businesses and bureaucracies. Is that because the people involved don’t recognize our reasons for coming? Because we file a million forms but never get the help we really need?

A gateway organization could help end this bureaucracy. It could rank government services and rank big businesses by how they address our reasons for coming.

Ending Inhumane Artificial Intelligence

Okay, one more. What do these have in common? Creepy advertising that follows you from site to site. The Terminator and Matrix movies. Elon Musk saying AI is an existential threat. YouTube’s attempt to get me to laugh alone.

They show a growing fear of tech platforms, artificial intelligence, and big data.

And perhaps this fear is accurate! Clearly, the wrong metrics are running our tech ecosystem. And furthermore, our machine learning algorithms don’t give us reasons along with their results, and that could make them untrustworthy.

Right now, machine learning algorithms are simple maximizers that study, predict, manipulate, or accelerate our behavior without regard for our reasons. How can we feel good about AI like that? ¹⁹

Consider the algorithm behind facebook newsfeed: even if it was trying to make us happy, would that be true cooperation? Unless we came to facebook just to be briefly happy, I don’t think so. There’s a way it’s doing something to us, not doing something with us.

This kind of simple maximizing AI is probably doomed to be irresponsible and unreflective in just the same way as our metrics-driven organizations.

But could we design neural networks that have inspectable long term values? That could report reasons for their actions?

It may not matter if the algorithm is just recognizing images. But it sure matters for book recommendations, or for newsfeed! It matters for any situation of cooperation with humans. Only when algorithms share reasons with us can they cooperate in the deep sense we’re used to.

A gateway organization could provide new metrics for evaluating these AIs and recommender systems. And that could trigger a flight to higher ground.

Beyond Capitalism

So that’s overconsumption, bureaucracy, and AI.

If the Internet sector changes its metrics, it won’t stop there. We might change the whole economy.

Apps and websites can be gateways organizations — like Yelp — for every other sector of the economy. People decide what they’ll buy, where they’ll live, what they’ll do with their friends, where to travel, and even which doctor to use… All using apps and websites.

It’s worth changing the whole thing. Often, the products and services that do best, are those that eat the most of our time or our money.

In other words:

We’re rewarding the parts of the economy that drive behavior and spending, rather than those with the highest benefits. ²⁰
Rewarding our biggest investments rather than our best returns.

The time and money we put in aren’t the best measure of what we get out. Because we didn’t come to spend money. That’s not our reason, that’s not our identity.

We have an the economy that amplifies our investments, our costs, our sales, our app downloads, our views. A cost economy, not a benefit economy.

What would it be like, if we could actually incentivize what we want out of life? If we incentivized lives well lived.

We all value different things. But our values, by definition, are our best measure of what’s worthwhile. Our values and commitments are here to be honored.

And by changing metrics in tech, we might be able to build an economy — and build products — which honors our values, commitments, and identities.

Measuring Reasons + Outcomes Together

Okay, well, I’d like to help everyone get started with this.

A first step is for organizations and algorithms to begin to recognize our reasons for coming.

So, we need to collect those reasons from users.

What kinds of reasons do we need to collect? One kind you might be familiar with, is logistical goals. Software design has focused on this, in the past. Software designers think of the user as having goals: ²¹

They want to rotate this photo
or send this message
or even, if it’s google maps, they want to get to California.

But mostly we don’t do things because of logistical goals like that: ²²

For instance, if you did something because you were bored or tired, those are reasons but they aren’t goals.
Or if you just like doing certain things, those aren’t exactly goals

and of course we often do things for other people: ²³

You might do something because you’re committed to someone,
or because you care about someone.

And that doesn’t mean you have a goal with them, or a goal for them. ²⁴

It turns out there’s some categories. Using those categories, we can create a tree of different kinds of reasons. That tree can capture the full breadth and variety of why people do what they do.

Do Users Have Reasons?

Some of these reasons are vague, some are shallow, some are really deep.

If we’re going to collect reasons from users, the interfaces have to work with the way the user understands themselves. There are many interfaces that can work: we can type our reasons, or we can swipe our reasons. We can also guess and check our reasons.

These interfaces can work whether reasons are vague or deep: Maybe I only know that I go to YouTube when I’m bored, or because I feel like a break. That’s fine. ²⁵

The reasons that result from all of this, they can be gathered up into a database where they can be queried and aggregated, and where the reasons behind the use of every product or service can be understood.

Could YouTube Have Reasons?

But the best thing to do with this data, is give companies new metrics. They need new ways to score themselves, and new ways to score their algorithms, so they can focus on serving their users for the reasons they come.

And the best thing would be if these metrics would change companies and their algorithms into reason-based maximizers, instead of simple maximizers. They could be responsible for their choices, they could evolve and grow, and they could be trustworthy.

What would it take? What would such a metric look like?

Here’s what we know:

For a team using this metric, choices would not just be about numbers. At least some choices would also be choices between different values that the team could have.
This metric would have to recognize the reasons users come to the product or service.
And more than that, it’d have to share those reasons — in other words, an organization using this metric would have to be completely aligned with those reasons. Would have to want those reasons to work out for the user in the long run.

So in this world, YouTube would be a reason-based maximizer, just like me. We’d both have identities made of values, commitments, and customs, and we’d both come to our transaction together for reasons.

I come to laugh with friends, to learn music, and to face my fears.

But what could YouTube’s values be? What could its identity be that would be worthy of trust, because w’d have shared reasons?

YouTube has to measure its own performance by whether my reasons for coming work out in the long run.

In other words, what YouTube want is for my my choice to use YouTube to be lastingly positive. We can use this idea to rewrite my reasons, as reasons for YouTube:

I come to laugh with friends. YouTube comes because it wants to be a lastingly good choice for laughing with friends.
I come to learn music. YouTube comes because it wants to be a lastingly good choice for learning music.
I come to face my fears. YouTube comes because it wants to be a lastingly good choice for facing fears.

If YouTube has these values, I can truly say it shares my reasons. With these values, YouTube deserves my trust. If this is how YouTube measures its performance, we are truly aligned. ²⁶

Now, users come to YouTube for many reasons. Which of its users’ values should YouTube make its own? The YouTube team could pick. It could pick what from its users goes into its own identity. That makes YouTube’s choices, not just about numbers, but also about YouTube’s values and YouTube’s identity. BAM! We’ve transformed YouTube into a reason-based maximizer. A reason-based organization that runs reason-based algorithms.

Making Reasons into Metrics

In order to monitor how its doing for its new values, then, YouTube needs to know whether it’s still regarded as a lastingly good choice for the reasons users come, and how it might do better.

So, users can tell them how their reasons are working out: ²⁷

Are you still glad you go to youtube for facing your fears?
Are you still glad you go to facebook to feel loved?

There’s a huge opportunity for business here.

Every business wants to spread by word-of-mouth. That happens when it helps people with what they came for.

By collecting data about the reasons people come, and how they work out for people, we can see whether this is happening, and tune products and services to make it happen more.

So, YouTube could see how much of their usage is regarded as a lastingly good choice — as Time Well Spent — and how much is regretted, or is no longer regarded as working well for that reason.

We could assess how YouTube is doing for all the reasons people come:

For each reason, is YouTube a lastingly good choice? Or do people come to regret their choice to come for that reason?

With this data, YouTube could steer itself in a new direction.

Whole Person Analytics

We could call this whole person analytics. We mean analytics about reasons and fulfillment, not just superficial behavior.

Whole person analytics has so much to add to how companies see their products and users.

For instance, traditional analytics focuses on funnels. But those funnels are oddly truncated. Here’s a funnel for Tinder — the user starts by installing it, launches the app with this frequency, does this many swipes, has this many matches and this many chats.

But when you think about it, this funnel is incomplete.

One thing it’s missing is why the user came. What if we knew why users were installing the app in the first place? Why were they launching it? Do they feel like it’s actually a good use of their time in those moments?

Do many users agree that there are better options when you’re bored?

And what happens for users at the end of the funnel? What if we could see how things are working out for them:

how much time they’ve invested in our app,
what returns they’re getting on that investment,
and how they feel about it, compared to other options for their time?

Understanding Motivations

With a slightly different view, we can even see the deeper reasons driving each interaction. There is an iceberg of reasons hidden beneath each user action.

Here, swipes and matches are driven by a desire to feel hot, just as much as they are driven by hopes for dates or for sex.

And we can see in the shading, whether these reasons are lasting reasons for people (in which case they’re green), or fleeting parts of their identity that they soon revise (in which case they’re red or yellow). When a reason hasn’t been thought through, or when it conflicts with other values a person has, it’s likely to be a short-lived reason. So here, it’ll be red.

We can also see how much time people spend in our product, and whether they’re getting a decent return, or whether they are discovering better options that take less of their time.

Here, the return on swipes for people who want dates is declining. That could be a big problem.

Next Steps

Organizations need ways to listen at scale to the people they’re supposed to be serving. So whole person analytics has huge value inside organizations and product teams.

So many companies would be interested in the reasons their users come, and in whether they are fulfilled in the long run.

I’ve used YouTube as an example, but think of the non-profits — like Wikipedia and Khan Academy. Think of the subscription services that are about using time well — like Asana and Pocket. Think of the organizations that lose out in an economy of clicks and views but could win big in an economy of time well spent and lives well lived — like MeetUp, WikiHow, and DIY.org.

Guerrilla Analytics

But what about the companies that aren’t interested? The companies that want to manipulate us, that put profit above serving their users?

I think we can involve them too, against their will.

Just like FDA with their evaluations of drugs, or Yelp with their evaluations of restaurants there’s an opportunity to trigger flights to higher ground on the internet.

To do so, we need to be able to run whole person analytics from outside the companies involved. We can collect data — about users, and why they come, and how it works out — at the platform level.

For instance, I’ve built a chrome extension called Hindsight — you can get it right now at the chrome app store. It asks you why you visit your websites, and how they work out for you. It shows when services that seem popular are actually sources of regret. ²⁸

And of course we could build similar functions into Android and iOS.

The other companies that I mentioned before — the ones that are part of the benefit economy — would want to support this.

Earlier in the talk, I said consequences were not the source of true cooperation, but that they could keep simple maximizers from betraying us. Maybe this data collection could create consequences that would keep simple maximizers in their place.

Impact

In any case, there will be flights to higher ground. Some companies will care about the reasons their users’ come and how they work out. And the economy will change.

Which companies will change their algorithms and organizations?

Even if it was only YouTube, it’d be pretty impactful. People would watch different videos and YouTube would launch different features. There’d be less laughing alone and more laughing with friends, more ukulele playing, more breakdancing.

That would be a good thing. YouTube accounts for 1 billion hours of people’s lives every week. And if Facebook switched, that’d be another 9 billion hours a week, including most of what people read.

But I’m optimistic that it won’t just be the 10 billion hours people spend on Facebook and YouTube. That it’ll actually be the $35 trillion dollars that individual consumers spend on everything they do.

Conclusion

We invented metrics-driven organizations and bureaucracies just about 150 years ago. Ever since we invented them, they’ve grown ever more powerful.

And individuals have become less powerful.

We’ve been struggling between large-scale endeavors, on the one hand, and really serving people, on the other. So far, large organizations haven’t done well for those they’re supposed to serve.

So lets see what happens when we listen to reasons, instead of just behavior. Maybe reason-based metrics and analytics can resolve this tension, can allow us to believe in organizations and algorithms the same way we sometimes believe in one another.

What lifestyles will be available to us, when we aren’t pressed towards consumption but instead are supported in our real values? What will work be like, when it’s based on shared reasons between workers and the organizations they join?

Time will tell.

Thank you.

Credits

Okay, I’d like to thank some people and then tell you what’s coming up in the next three talks.

First, the idea of giving these talks emerged in conversations with Bret Victor and Patrick Collison.

The development of the talks was funded by a grant from Stripe, and many people in Bret Victor’s lab, CDG, have supported me along the way.

The philosophical ideas about reasons, identity and trust are founded in the work of two philosophers — Ruth Chang at Rutgers and David Velleman at NYU. My argument about trust descends from a somewhat weaker argument of Velleman’s, and both Chang and Velleman have similar models of how identity evolves during choicemaking. To read their papers, check out the bibliography online.

It was Nobel-prize winning economist Amartya Sen who best framed the limits of simple maximizing, with other key contributions by Allan Gibbard and Isaac Levi. Again, check the bibliography.

And finally, thanks everyone who reviewed this talk and gave me feedback and encouragement.

Next Talks

So this was the second talk out of four, in an ongoing series on human fulfillment and tech.

The first talk was on Designing for Agency — about how to design software that’s empowering rather than distracting.
This was the second talk.
The next, will be about Transformative Social Gestures — how other’s lives can transform our lives, and what that means for social networking.
And the fourth will be about Platforms and Dignity — about redesigning operating systems so they support the full person using them and keep apps in check.

Contact Info

If these topics appeal to you, get in touch. All are areas of active research at our lab in Berlin, and we’re building community around them. We also want to build a training program for the industry, and we’re looking for partners.

Thanks again. Be reasonable to one another.

Footnotes

For an overview of some of the problems that result, see part 2 of this talk, and James Scott’s Seeing Like a State.
The exact metrics from CouchSurfing are documented on these archived pages from the CouchSurfing wiki.
Here’s one way to think about how values, reasons, and identity interrelate:

Some of our reasons for action are fleeting, and these are not values. For instance, “having to pee” is a good reason to go to the bathroom, but it’s not a value. The less-fleeting reasons are our values.

It’s probably not important whether the idea of values or the idea of reasons is more primary—i.e., it may be just a matter of definition whether our values give us reasons for action, or whether our reasons confer value on things (like the bathroom, above).

Besides values and fleeting needs, some other types of reasons are goals, commitments, intentions, and customs. There’s a full taxonomy at the beginning of part 3.

I never use the term reasons in the sense of “the reason something happened”, i.e. as a cause, but rather in the sense of “a reason to act.” In other words, reasons justify an action for us. It’s not important whether reasons cause action. To say something about causation would be to say something about the human being as a mechanism and specifically about the brain and ultimately about the laws of physics. It’s a cartoon view to say that a thing like a reason (or a brain chemical) causes some observed behavior in such a complex system. That said, we do mostly make choices for reasons, in the sense that the reasons are there for us during and before the choice (see footnote 25). And the triple nature of the word “reason” suggests that in our direct experience, it feels like the process of rationalizing during choice sometimes causes us to do things.

If it doesn’t make sense to talk about causation in the brain, it probably also doesn’t make much sense to talk about motivation as if it were one thing. While our reasons may or may not motivate us, we have a sense that they should and that it’s somehow spooky when a strong reason for action doesn’t motivate us, or—worse—when we are motivated to do something we have absolutely no reason to do. So if there’s any objective sense in which to talk about motivation, it seems clear that reasons for action are bound up in how it operates.
Here, the actions of a simple maximizer correspond to those assumed of people and organizations in standard rational choice theory (SRCT), which forms the theoretic foundation for microeconomics. Academics have found many inadequacies in SRCT. In this talk, I focus on one—that agents who operate according to SRCT don’t create a functional society.
There are many conceivable ways of weighing reasons during choice. For a sort of catalogue, see Isaac Levi’s Hard Choices (excellently reviewed by Sen), and Ruth Chang’s Making Comparisons Count. What’s important for the analysis here is: (a) that reasons often bear on an option decisively, rather than additively; (b) that reasons can be disentangled in such a way that we can roughly see how changes in our identity will change reasons and choice evaluations; and (c) that reasons consider aspects of the choice besides just its possible outcomes.
As a first approximation, mathematical formalists could think of identity as a set of evaluation functions. But the components of our identities — whether they are described as our values or our reasons for action — interrelate and have a deep structure that bears on their evaluation. For one hints about how values interrelate, see Chang’s Making Comparisons Count and Putting Together Morality and Wellbeing. For a model of how they evolve, see Velleman’s Practical Reflection.
Some philosophers and economists have tried to mod SRCT in such a way that choices are made on larger timescales. It’s not clear if there’s coherent way to do this that doesn’t end up with something quite similar to a Reason-Based Maximizer.
Social science works by making simplified models of how humans act. It’d be simplistic to say that people *are* reason-based maximizers (or simple maximizers). We act more like simple maximizers when making some choices (e.g., choices about probabilistic outcomes over numeric payoffs) and reason-based maximizers when making other choices (everything from which website to visit to which career path to choose). Models of human behavior based on simple maximizing try to connect themselves to human nature via the doctrine of “revealed preference.” This doctrine makes an extreme claim. It’s not just that our choices reveal our preferences (they obviously do), but that they express our preferences in a particularly straightforward manner because each choice is a sensible gamble, betting towards a payoff that would be a positive step in assembling a ideal package of goods or services which is of highest utility for us. Amartya Sen obliterated this notion in his 1973 classic Behaviour and the Concept of Preference, but economists haven’t arrived at a replacement for the doctrine in the 43 years since. Perhaps the notion of reason-based maximizing would help.
This is the classic excuse of the bureaucrat. I’d argue that this happens because the bureaucrat finds himself in a simple maximizing regime. I.e., it is practices like quarterly reports, OKRs, and KPIs which have driven the bureaucrat to this stance. See Neil Postman’s Technopoly, Chapter 7 for how this works.
Ruth Chang would say these “real choices” are what we get when the options are “on a par” (e.g. in, The Possibility of Parity).
Another way to look at this process is that it’s about navigating towards more comprehensive values. See “Putting Together Morality and Wellbeing.”
Of course, we could imagine adding such functionality to simple maximizers. Such a machine would have an evaluation function with the swappable terms that can be added and removed. If those terms end up mapping to concepts like human values, or reasons for action, then bam! you’ve turned a simple maximizer into a reason-based maximizer.
Read about the iterated prisoner’s dilemma or search the literature for reputation systems and the prisoner’s dilemma if you’re new to these ideas.
Note that changing the math actually doesn’t get rid of the prisoner’s dilemma, it just makes it less common. It doesn’t explain why humans cooperate often, even when experimental conditions limit consequences, and limit caring, and thus keep the prisoner’s dilemma structure intact.
This argument comes from David Velleman’s The Centered Self. His argument wasn’t received well. I developed the reason-based maximizer model to try to place it on firm ground.
I talk about what makes reasons fit together in part 3 of this talk.
In the Theory of Practical Reason, this is often called a Shared Intention.
Economists have tended to focus on monopoly and rent extraction as the dangers of big business. Maybe the real danger is that through growth it has historically become impossible for organizations to address the nuanced diversity of reasons people come.
Perhaps a way to think about unsupervised neural network models is that a network is discerning values.
J.M. Keynes foresaw most of this! From Keynes’ Economic Possibilities for Our Grandchildren:

“I see us free, therefore, to return to some of the most sure and certain principles of religion and traditional virtue — that avarice is a vice, that the exaction of usury is a misdemeanour, and the love of money is detestable, that those walk most truly in the paths of virtue and sane wisdom who take least thought for the morrow. We shall once more value ends above means and prefer the good to the useful. We shall honour those who can teach us how to pluck the hour and the day virtuously and well, the delightful people who are capable of taking direct enjoyment in things, the lilies of the field who toil not, neither do they spin.”
For a classic example of this view, read Alan Cooper, The Inmates are Running the Asylum
Velleman’s The Centered Self has a little starter taxonomy of reasons, which I’ve developed further in this section, and in software.
Chang has written extensively about commitments.
An argument in Velleman’s Beyond Price suggests that you mustn’t have a goal for anyone you love, or you’ll trample on their agency which is the core of what you love.
There is a common view that, unless people are very conscious or self-reflective, they likely do things for no reason at all, and merely make up reasons after the fact to justify their mindless, involuntary, twitch-like actions. This view seems to come from a strain of behaviorist psychology that views people as addicted or habituated or socially programmed or driven directly by impulses or by brain chemicals (“desperately seeking dopamine”) or by evolutionary imperatives around status or reproduction. The view that we only justify our random actions in hindsight is bolstered by psych studies where people are unreliable witnesses, or where we narrate our own actions in seemingly arbitrary ways.

But our capacity to make up reasons in retrospect doesn’t indicate that that we don’t have reasons in the first place. And there is ample evidence that we do.

The view that people do things for no reason appeals mostly to those who’ve never looked into the economics of choice and addiction, who’ve never considered how alarming it is when we actually do something for no conscious reason (like sleepwalking or throwing up or responding to laughing gas), and who’ve never introspected about the reasons behind their own “bad” choices.

If you look into the economics of choice and addiction, you’ll find that so-called “addiction” is indistinguishable from rational actions taken in a tough situation (Stigler & Becker, Theory of Rational Addiction). If you look into the philosophy of action, you’ll find a coherent view that, not only do we make up reasons for our actions, we also make up actions for our reasons (Velleman, Virtual Selves). If you introspect about your own “bad” choices, you’ll find that even a “bad” choice to eat junk food or watch TV or do something you’re conflicted about comes from a “good” reason, like wanting to stop feeling anxious, or to do something about your loneliness, or to get your mind off something you don’t know how to face. The truth that emerges is this: that people of all levels of self-reflection and articulacy do things for good reasons, and that psychological theories that paint people as addicted, habituated, socially programmed, or driven directly by brain chemicals are a dangerous, elitist, and disrespectful kind of bullshit. Why are these theories dangerous? Because it’s only if people have reasons that you can possibly help them with what they want to do. If you imagine that there’s nothing behind people’s actions, that they don’t have anything noble that they hope for, that there’s nothing their actions are about… the best you can do is project your own goals for them (happiness, health, mindfulness, productivity, etc) and you needn’t have any concern about whether they want those things for themselves.
One thing that’s not in this talk is an equation for reducing information about users’ reasons and whether they are working out to a scalar number for YouTube to score how well it’s doing. There are multiple ways this could be done and I’m not quite ready to advocate for one of them. One approach would be for YouTube to try to measure the total amount of time, across all their users, that resulted from choices to use YouTube that those users are lasting glad they made. In other words, YouTube could try to measure the total Time Well Spent they’ve made possible. This is a good start, but it doesn’t address distributional issues, either internal to users (is the first hour of Time Well Spent worth the same as the 40th) or across populations. To read more about distributional concerns, check out Sen, Equality of What, and Alan Gibbard on Interpersonal Comparisons. And it’s likely that, in the near future, the Center for Livable Media will announce a recommended way of scoring organizations or products based on user reports about reasons and outcomes.
Ideally this kind of review process becomes part of our tech platforms in iOS, Chrome, etc. See my chrome extension announced below.
We plan to make a leaderboard of sorts. Contact us at livable.media if you’d like to help.

Bibliography and Further Research

I. Values and Reasons in the Individual

What are values? What are desires? What are reasons for action?

Velleman, David (2006), The Centered Self. http://www.academia.edu/2048271/The_Centered_Self
“What’s needed is a conception of practical reasoning that has a role for our sense of identity, which might in turn explain our capacity for credible commitments.”

How are values structured, and how do we compare options with respect to them and make choices?

Sen, Amartya (2004). Incompleteness and reasoned choice. Synthese 140 (1–2):43–59.
Chang, Ruth (2014). Making Comparisons Count. Routledge. [BOOK]
Sen, Amartya (2000). Consequential evaluation and practical reason. Journal of Philosophy 97 (9):477–502.
Chang, Ruth (2004). All things considered. Philosophical Perspectives 18 (1):1–22.
Chang, Ruth (2004). Putting together morality and well-being. In Peter Baumann & Monika Betzler (eds.), Practical Conflicts: New Philosophical Essays. Cambridge 118–158.

Advantages of Reasons Over “Irrationality”, “Cognitive Bias” etc

Becker, G. S., & Murphy, K. M. (1988). A theory of rational addiction. The journal of political economy, 675–700.
Stigler, G. J., & Becker, G. S.. (1977). De Gustibus Non Est Disputandum. The American Economic Review, 67(2), 76–90.
Philip Nelson (1970), “Information and Consumer Behavior”

Advantages of Reasons Over Utility

Sen, A. (1973). Behaviour and the concept of preference. Economica, 241–259.
Velleman, David (2006), The Centered Self. http://www.academia.edu/2048271/The_Centered_Self
Chang, Ruth (2014). Making Comparisons Count. Routledge. [BOOK]

II. Values and Reasons as We Live

How do values evolve in a person?

Velleman, David (1989). Practical Reflection. Princeton University Press. [BOOK]
Chang, Ruth (2013). “Commitments, Reasons, and the Will.” Oxford Studies in Metaethics 8 (2013): 74–113.
Chang, Ruth (2009). Voluntarist reasons and the sources of normativity. In David Sobel & Steven Wall (eds.), Reasons for Action. Cambridge University Press
Velleman, J. David (2008). Beyond price. Ethics 118 (2):191–212.
Betzler, Monika (2012). The Normative Signiﬁcance of Personal Projects. In Michael Kuhler & Najda Jelinek (eds.), Autonomy and the Self. Springer 118–101.

Reasons and Outcomes (Consequences, Rational Regret; Reviewing States, Actions, Motives, and Processes)

Sen, Amartya (2000). Consequential evaluation and practical reason. Journal of Philosophy 97 (9):477–502.
Betzler, M. (2004). Sources of practical conflicts and reasons for regret. Practical conflicts: New philosophical essays, 197.

II. Values and Reasons across Individuals

Aggregations (Interpersonal and Interpersonal)

Sen, A. (1980). Equality of what? (Vol. 1, pp. 197–220). na.
Gibbard, A. (1986). Interpersonal comparisons: preference, good, and the intrinsic reward of a life. Foundations of social choice theory, 165–194.

Contents