The Bias and the Noise or Why Our Decision Making Can Make No Sense
Eight investment managers were given a set of six sample cases and told to use their years of expertise to decide which cases deserved investment. The psychologists running the program asked the executive team that had hired them to guess the percent difference in decision making between the managers.
The executive team guessed 10 percent. And this is on average what most guess.
The real difference was fifty. 1 out of 2 decisions between the managers was arbitrary and unique to that individual. Executive management was not happy.
On hearing this story you may, like me, jump to the conclusion that bias influenced the managers. But according to David Kahneman, the Nobel-prize winning economist and author of Thinking Fast and Slow, bias was designed out of the study i.e. the managers were similar enough in behavior, background etc for that not to be the case.
So why the radical difference in decision making? It’s NOISE. Simply defined as a set of variables too arbitrary to help us identify a cause (this is me trying to summarize a much more articulate definition from Kahneman.) We assume bias when it’s a decision that could be influenced by time of day, or how someone feels or what they had for breakfast.
I had a chance to hear Professor Kahneman share this story last week, and it warped my worldview. My goal with OpenUp has been to get to causality: if we have enough data we can identify why people are doing what they are doing. And the logical extension of that is that the data will show rational decision making that we can then turn into a business strategy to grow audience or profits or make more widgets.
But we’re not rational.
My next mattress will be a Casper mattress. I’ve never been targeted by a Casper ad. I’ve slept on a Casper mattress and thought it was OK. But I really like the cartoons in the subway. I know they are the “cool new” mattress company. I might say because the founders went to Brown, but a founder of EVE went to Cambridge and I know him to be a cool guy too.
The variables:
v = cool matters
w = random good feeling alumni affiliation
x = like Casper cartoons on subway
y = never seen online advertising for Casper
z = slept on Casper bed and thought it was OK.
How do we find the ROI?
v + w + x = making me feel good
y = 0 (I never saw online ads, wasn’t targeted)
z = experience is good enough
So how do we get me to buy Casper?
3 x [Making me feel good] + 0 + [Okay is good enough] = 1 sale of a Casper mattress
This is an equation unique to Ashwini. This equation is filled with noise. This equation probably doesn’t make sense and need many more variables to understand the buying decision. Maybe there are enough quirky people like me that Casper can identify to build an ROI equation. Or maybe it’s a decision unique to me and on the day that I decide to buy, I’ll feel differently and buy a competitor. There’s a 50% chance I will.
We just don’t know. But I strongly believe that the more information we have, we can at least begin to smartly hypothesize.
This is why OpenUp asks users to share their data. Why we link behavior with attitude. Because we need to weed through the noise. It’s always going to be guesswork. Because we are selling to irrational actors. But doing so gives you at least a 50% shot at success.
p.s. Like this story? Follow us on Twitter
p.p.s. Can we get a tweet or a share? We would love you forever
