ANSWERS, NOT DATA. Why some marketers succeed with analytics and others don’t. (A Medium book). Chapter 3.
“If you don’t know where you’re going, any road will take you there.”
The Cheshire Cat (paraphrased), Alice’s Adventures in Wonderland by Lewis Carroll
F or marketers, especially when dealing with analytics, knowing what is the real problem you are trying to solve for is a critical activity you’ve got to get right. Yet like most people, including me, at some point, you’ve found yourself halfway through a marketing or analytics project and suddenly thought, this is stupid, we are trying to solve the wrong problem? Just like Alice, in your situation, since you don’t know the real problem to solve, the Cheshire Cat would tell you that any answer will do. Get the problem wrong, and nothing else you do about it will be of value.
But it doesn’t have to be that way. In this chapter, we’ll cover how probing and asking “Why?” is the most powerful question you can ask to uncover the real root problem to solve, versus the surface effect that we too often mislabel as the problem. Let’s start by unpacking a significant barrier to proper problem definition and lack of probing.
Why people struggle with probing and problem definition
I n my experience, people who fail at the probing and problem definition process fall into two groups. One group simply wants to get to build a solution as that is the fun stuff. Working on producing something is filled with creative expression, and in that space, our bodies will produce dopamine, serotonin, oxytocin, and endorphins, the quartet of chemicals responsible for our happiness.
The other group wants to avoid the effort of probing and defining the problem since critical thinking is hard. It involves being skeptical and challenging assumptions, rather than merely memorizing facts or blindly accepting what you hear or read. That can wear a person down. Who among us hasn’t had the feeling of mental exhaustion following a deep cognitive exercise such as a test or a similarly grueling mental marathon?
But without the rigor to define the problem, then time and money can all be lost, chasing the wrong thing. Have you ever seen an analytics project go down one road and later on realize it told you nothing? Have you seen an innovative approach to deliver a breakthrough product only to find out that customers didn’t so much care about solving that problem?
I’ve been in many meetings where the business is having a downturn, and without even thinking through the problem, a senior executive will shoot from the hip, saying something like “Let’s do such and such.” When in truth, if they had taken even just a few breaths and assessed what was going on by probing the surface problem and asking why, they might have found out that the issue was something entirely different, and their idea wouldn’t then make sense. I’ve seen this a lot, and I bet you have too.
There’s an often-cited quote attributed to Einstein that no one knows for sure if he said it, but it captures well the importance of properly defining the problem.
“If I were given one hour to save the planet, I would spend 59 minutes defining the problem and one minute resolving it.”
Albert Einstein? Maybe, maybe not.
Spending time defining the precise issue is, therefore, essential for success with analytics, marketing, business, and life. And it doesn’t always take much time to define the problem either. Sometimes it just takes a bit of thinking and asking a few simple but powerful probing questions.
Why do you say it’s too cold?
When I worked for Siemens, I was an adjunct faculty member in their world-renowned sales development program. A few times a year, I taught a 3-day class on selling to healthcare facilities, which included a module on problem identification. In class, I used this scenario.
Imagine your office maintains the heating and air conditioning at a local ambulatory care facility. An influential Doctor is complaining that her office is too cold ever since Siemens put in new temperature controls. She contacted the Facilities Director at the central hospital and he, in turn, called you to check it out since you have a maintenance contract on the building.
I’d then begin to question the class about the problem, and what to do about it. Many students usually and quickly offer up solutions such as increasing the heat or airflow in the room. But is lack of heat the real problem? Well, maybe, and maybe not.
The next step in the class is I share that you the manager met with the Doctor. You saw that the room thermostat is working correctly. The room temperature is 68 degrees, as is the design.
I then ask the class what else can you do? At that point, I usually hear more solution ideas as if it is a creative thinking session. This behavior will generally go on for a while until a student finally says something like, “How about asking the Doctor, ‘Why do you say it’s too cold?’”
Bulls-eye! When “Why” is asked, it starts to unlock the truth. Why is a probing question, and the most powerful question you can ask when you’re looking to find the root cause of a problem, or surface the real unarticulated need when doing market research.
I’ll then help the class by saying the Doctor responds as follows. “Well, because when I put my hand over the heating unit, the air flowing out feels cold.”
At that point, the light goes on for the students as they know, being in the HVAC business, that hot air flowing over your hand will feel colder than the actual air temperature. The issue was two-fold.
First, the Doctor believed (there we go back to believing and not knowing) that the room was cold because of what she perceived as cold air coming out of the air register, and because of that, she thought the room thermostat was wrong. I remind the class a mechanic will usually put a thermometer in the air register and show the Doctor the temperate is fine.
The second issue is the students jumped into premature solution mode looking to be the hero with the best technical solution without spending time asking probing questions to tease out what the problem might really be. All they had to do is first ask: why?
Like everyone, I sometimes short-change probing and the problem-framing step, especially when I’m tired, rushed, have too much on my plate, am not self-aware, or when my ego gets full of itself as I think I’m right and I allow it to rule. When those things happen, and I get into fixer mode too fast, I often later regret it. When I miss the problem framing step and go into Bob The Builder mode, it usually leads to less than ideal outcomes.
But when one properly inquires about the problem and probes a bit, magic might happen. The more you do an in-depth evaluation and probe, the more you’ll identify different aspects of the problem. That, in turn, can sometimes deliver radical improvements; and even spark solutions to dilemmas that have seemed intractable for a long time.
Asking “Why?” is the easiest and most powerful way to probe and uncover the truth, especially when you ask it at least five times.
The Five Whys
K iichiro Toyoda considered Japan’s Thomas Edison, was a Japanese businessman born in 1894, and the son of Toyoda Loom Works founder Sakichi Toyoda. Kiichiro’s bold rebellious decision to change Toyoda’s focus (and name spelling) from a leading automatic loom manufacturer into automobile manufacturing created what would eventually become the global powerhouse of modern fame today, Toyota Motor Corporation.
Kiichiro liked the idea of asking why so much that he led the creation of Toyota’s Five Whys process, its world-renowned scientific approach to uncovering the right problem by repeating why five times. With almost a century of use, this approach has become a vital tool within market research, product development, Lean Startup, Lean Manufacturing, Six Sigma, and all types of business operations evaluation. It also helped propel Toyota’s legendary quality.
Don’t be misled. Five Whys isn’t just casually asking why five times. It is an iterative examination technique used to uncover the cause-and-effect relationships underlying a particular problem. The primary goal of Five Whys is to determine the root cause of a defect or issue by repeating the question “Why?” The key is that each answer forms the basis of the next question. Therefore, the asker has to carefully listen to the responses to build on to the next why. For example:
Here’s a surface problem: The online conversion rate from paid search is off from last year. So what’s the root cause?
1. Why? The share of traffic from lower converting mobile devices is up.
2. Why? We’ve spent a much greater share of Google PPC and PLA spend on mobile than planned.
3. Why? We haven’t been tracking this as carefully as we ought too.
4. Why? We are not using a disciplined repeatable approach and checklist in looking at weekly results from our online ad agency who is managing our account. I’ll come back to checklists in a future chapter.
5. Why? The person who is leading paid media is more so a brand person and isn’t a strong quant person with the numerical skills needed with managing online advertising.
We’ve now found a possible root cause, and closer to, if not likely the real problem to solve. The questioning for this example could be taken further to a higher level. Still, five iterations of thoughtful asking why, are generally sufficient to get to a root cause or the real problem that needs solving.
You could certainly apply a solution to whys #1 to #4, and that might help short term. But there is still the underlining root cause of #5 that needs to be addressed to assure the long term success.
A fundamental rule in Five Whys is for the trouble-shooter to avoid opinions and logic traps and instead trace the connection of causality in straight increments from the effect through any layers of notion to a root cause that still has some relationship to the initial problem. That sometimes takes a little time as you may get stymied when you ask why. You might need to dive into a particular why and explore it before you can articulate the next why. Therefore don’t expect you’ll have results as clean and fast as the online ad example above. But using this technique will ultimately help you get to the root issue to work on.
In those situations, when the next why isn’t apparent, you can augment your probing using the word why, with a few other choice probing words. Let me share a personal example.
What to do when you’re stymied with asking “Why?”
A friend is the CEO of a $90 million private equity-owned building supplies business. A few years ago, he asked me if I could speak with his head of merchandising, James (not his real name). James was advocating a significant price reduction as he though pricing was the cause of sales being down. My CEO friend wanted my opinion.
As I recall, when I spoke with James, the gist of the conversation went something like the following.
Tim: (1st why) “I heard you want to do a price reduction. Why is that?”
James: “Our sales are down about 3%.”
Tim: (2nd why) “Let’s put to the side whether or not a price reduction would stimulate enough demand to offset the lost margin. Let me ask you, why are sales down?”
James: “It’s not clear to me. Everyone has different opinions. I’m not sure but I think it is pricing related.”
Tim: (3rd why) “Why do you want to implement a price reduction if you don’t know what the real issue is?”
James: “That’s a good point. When we promote a sale though, we normally get a bump. What do you think we should do?”
Tim: (I got stymied on my last why, so I try “Are”) “I have no idea. But let me ask you some more questions. I believe you have four main product categories. Are they all down?”
James: “Pretty much.”
Tim: (Still stymied so I try “How”) “How else can you cut your business? How about vendors. How are they doing? Are they all down equally?”
James: “We have a bunch of smaller vendors that are down a lot.”
Tim: (Got back into why flow. 4th why) “Why do you think that is the case?”
James: “We did a SKU rationalization project, and narrowed down the assortment to only two main suppliers per category, and one private brand on some categories.”
Tim: (I’m stymied by his answer so I try “How”) “How much are sales down due to that?”
James: “I don’t know.”
Tim: (Back to why. 5th why). “Why haven’t you looked into that?”
There was silence on the telephone. Now we had uncovered something James could look into, that candidly he ought to have done already. But he didn’t know about the Five Whys, which to some extent was the real root of the problem.
James did the analysis over the next few days and found out the lost sales were close to $5 million. Based upon the company’s growth, when you took that away, the core was up about 2%. James realized that overall pricing was not the real issue but rather sales were down due to discontinuing too many items and no one took the time to consider that.
Using Five Whys we had found the true reason why sales were down. Maybe a promo was still needed, as the same item sales were up only 2%, but maybe a sale wasn’t the best thing to do. Regardless it was clear the bigger problem was SKU rationalization and not base pricing.
Finding the real problem is critical because if you’re going to apply a solution to something, you better know the real problem you’re trying to solve. Otherwise, any answer will do, and that answer might be way off.
However, all problems are not as linear as James’ situation. Sometimes uncovering the core problem takes a more thoughtful and in-depth evaluation, and application of Five Whys and probing.
Worst to first
I n the 1990s, the Oakland A’s were one of the worst-performing teams in baseball. They didn’t have endless sums of money to pay for the best players, and that reflected in their record as between 1993–1996 they won less than half of their games. In 1997 Billy Beane became their General Manager, and the team’s performance started to improve, to the point that in 2000 they won the American League West title. Then in 2001, they earned a wild-card playoff spot into the Major League Baseball playoffs.
The A’s did this with the second-lowest player budget in baseball at the time, of $39,722,689 per year, which was about 30% of the league-leading New York Yankees payroll of $114,457,768.
After the 2001 playoffs, the A’s lost their three-star players Johnny Damon, Jason Isringhausen, and Jason Giambi to free agency and teams with deeper pockets; the Boston Red Sox, St. Louis Cardinals, the New York Yankees.
Damon signed a four-year, $31 million contract with the Red Sox for about a million dollars more a year than he made in Oakland. Jason Isringhausen doubled his salary signing a $27 million, four-year contract with the Cardinals, And Giambi beat them both, increasing his annual salary by about $13 million a year as he signed a 7-year $120-million deal with the Yankees.
In a matter of days, the A’s were once again looking at becoming a losing team. What then is the problem to solve? Is it the A’s need to rebuild the team, or do they need to first address a more profound problem as they look for players?
On the surface, it seems like they just need to replace three of their star players during the Free Agency period. That’s easy to define, but it doesn’t explain the cause as to why they lost the players, which is the real problem they need to solve.
Let’s apply the Five Whys, starting with the A’s lost three key players to three wealthy teams. Thanks to another Micheal Lewis book, Moneyball, we have an insider vantage point about why.
Why did the A’s lose their star players? (1st why) Because baseball has changed. Just five years earlier, the highest payroll in baseball was $52M. The A’s could compete with that. But now there are big-market, big-money teams who have massive resources to bid for players. The A’s aren’t one of those, which led to others outbidding them.
Why aren’t the A’s financially strong? (2nd why) Because the A’s don’t have the attendance, broadcast revenue, or deep pocket ownership to compete in a bidding war. They are a relatively financially weak team and will continue so. But they keep trying to compete with big money during Free Agency, and they get outbid.
Why do the A’s continue to lose in Free Agency? (3rd why) Because the A’s, especially the scouts, continue to think and try to play the player personnel game like the big market teams where they focus on finding and buying star players. They try to select players as the Yankees do, however, the Yankees, Red Sox, Cardinals, and all but one other team in MLB can always outbid them.
Why do the scouts continue to operate this way? (4th why) Because the only way the scouts know how to find players is to use the approach they’ve used for years. They don’t believe there is another way.
Why don’t the scouts know any other way? (5th why) Because no one showed the scouts a different way to think about how to find undervalued players with a limited budget, players that the big money teams wouldn’t look for since they have the money to buy the perceived best the traditional way.
Now we’ve gotten to the root of the issue. Billy Beane and the A’s have to think differently. They can’t keep playing the game the way others are. Said another way,
“The world that we have made as a result of the level of thinking we have done thus far creates problems that we cannot solve at the same level as the level we created them at.”
Attributed to Albert Einstein, but maybe Ram Dass
A different approach, a different game, a different mindset, means you look where others aren’t looking. Much like Dr. Michael Burry, the A’s have to become value investors and look for the truth, being curious, getting in the weeds, and find patterns others are missing, so that they can find undervalued players.
Beane has a $40 million budget to invest, and as he looks around his comparable to stocks and bonds, are player salaries. The prices are well beyond his means. He needs to find value, underpriced players. But the scouts don’t know how to do that.
What a value investor does is use a lot of statistical data and analysis to help them uncover value. Burry didn’t throw a dart against a board and decide to bet against the mortgage market. He did massive amounts of statistical research.
Luckily for Beane, he was a fan of baseball writer and statistician Bill James’ theories of scientifically analyzing and studying baseball through the use of statistical data in an attempt to determine why teams win and lose. Billy Beane decided to give James’ ideas a try. But before he came to that conclusion, he had to ask why?
Once he had his answer to what is the real problem, Billy hired statistician Paul DePodesta, and together they embarked on an analytical approach to creating a winning team that, in the end, changed baseball and all professional sports forever. They used analytics to create a winning professional sports team.
“We are card counters at the blackjack table. We’re going to turn the odds on the casino”
Billy Beane, Moneyball (Movie)
And at the core of Billy’s card counting were Regressions and R-square. What I call marketers’ two best friends. Let’s explore them.
Stay tuned for Chapter 4 coming October 1st.