Analytics Alone Are Not a Solution
This article first appeared on the Proper Tack newsletter. If you like what you see here, sign up to get more articles like these as they are published.
It’s rare that I get worked up over an article about marketing and analytics. After all, these are not life-or-death situations, and someone else doing things poorly only makes it easier for smart people to succeed.
And yet, this recent HBR article on the “failure” of marketing analytics set me off.
I was honestly excited for this article. There are a number of ways that I think organizations use analytics poorly (turning metrics into business goals, restricting investment into people, over-reliance on algorithms, too much focus on Type-I errors and not enough on Type-II) and I was ready to see how other smart people are tackling these challenges.
Since this was published in the Harvard Business Review I figured I was in for a thoughtful and well-researched article.
I was mistaken.
The most recent results from The CMO Survey conducted by Duke University’s Fuqua School of Business and sponsored by Deloitte LLP and the American Marketing Association reports that the percentage of marketing budgets companies plan to allocate to analytics over the next three years will increase from 5.8% to 17.3% — a whopping 198% increase.
This is a very scary number to me. The only way this makes any sense is if that budget increase includes the cost of headcount for people to analyze, interpret, and strategize off of the results that come out of the various analytics platforms.
I am a vocal proponent of Avinash Kaushik’s 10/90 rule that says you should spend 10% of your budget on tools, and 90% of your budget on the people who are tasked with maximizing the impact of those tools. This was originally meant for web analytics, but I think it applies to any situation where you are relying on data to help drive decision making. If the software cost is much more than 10%, you should think very hard about why.
However, data grows on its own terms, and this growth is often driven by IT investments, rather than by coherent marketing goals. As a result, data libraries often look like the proverbial cluttered closet, where it is hard to separate the insights from the junk.
Absolutely. This is the bane of many an analyst’s existence. I would guess that half of a good analyst’s time is spent collecting, cleaning, and processing data from disparate sources before actually being able to start analysis. There are some tools that have improved this, and having a coherent strategy from the start of data collection can be a life changer, but this is just a major hurdle everyone experiences.
It’s also another reason why organizations should spend the money on people to think through and design those strategies.
(note that at this point I’m still super on board with the authors of this article)
Perhaps worst of all, data is often not causal. For example, while it is true that search advertising can be correlated with purchase because customers are in a motivated state to buy, it does not follow that ads caused sales. Even if the firm did not advertise, consumers are motivated to buy, so how does one know whether the ads were effective?
::record scratch::
Wait, what? A tenured Professor of Marketing who serves as the Executive Director of the Marketing Science Institute collaborated with the Editor-in-Chief of the Journal of Marketing and the result was asking “how do we know if ads were effective?”
May I suggest testing? Maybe something as simple as holdout panels? These have been a strategy for decades, way before complex digital analytics were a thing. You run ads in one geography, you don’t run ads in another, and you see if the sales growth is significantly different between the two.
This is an elementary tactic that doesn’t solve every attribution question (and believe me I have LOTS of attribution questions) but just blithely questioning whether marketing works is mind-blowing to me.
Companies should do two things to harness the power of analytics in their marketing functions. First, rather than create data and then decide what to do with it, firms should decide what to do first, and then which data they need to do it. This means better integrating marketing and IT, and developing systems around the information needs of the senior management team instead of creating a culture of “capture data and pray.”
I totally agree with this. Companies need to know what business goals are important to them, what steps customers take that move them towards those business goals, and then what data needs to be collected to measure those interactions.
But the title of this article was “Why Marketing Analytics Hasn’t Lived Up to Its Promise” and so far the answer seems to be “because marketers don’t have a coherent strategy for how to use them.”
Second, companies should create an integrated 360-degree view of the customer that considers every customer behavior from the time the alarm rings in the morning until they go to bed in the evening. Every potential engagement point, for both communication and purchase, should be captured. Only then can firms completely understand their customers via analytics, and develop customized experiences to delight them.
If you give up on reading the rest of this email, at least read my next few sentences.
The argument the authors are making here is bananas. This is the ultimate embodiment of the principle “perfect is the enemy of good.” Only now they’ve misinterpreted perfect to be “collect every piece of data possible” rather than “collect what is important and allows you to make decisions.”
If you want to build an ecosystem that tracks everything listed above I hope you have a team of 20+ analysts whose are solely focused on reporting, interpreting, and strategizing off of this data. Collecting more data through more integrations means that you need more headcount to see value from it. Companies should not rely on data collection in lieu of data analysis.
The CMO Survey also found that only 1.9% of marketing leaders reported that their companies have the right talent to leverage marketing analytics. Good data analysts, like good data, are hard to find. Sadly, the overall rating on a seven-point scale, where 1 is “does not have the right talent” and 7 is “has the right talent,” has not changed between the first time the question was asked in 2013 (Mean 3.4, SD =1.7) and 2017 (Mean 3.7, SD =1.7).
I would wager the actual problem isn’t that companies lack have the “right” talent, but that they have chosen not to hire enough talent. This isn’t a mythical man month where more people just clog up the process.
Drawing insights from data means analyzing results, coming up with hypotheses, testing those hypotheses, and then developing strategies off the ones that work. The biggest constraint for almost all organizations is bandwidth on identifying and testing hypotheses, and that is most easily solved with more smart people working on hard problems.
Companies need to better align their data strategy and data analyst talent to realize the potential that analytics can bring to marketing managers.
…
For example, a marketer coming to a data analyst asking questions about driving conversions might not realize that there’s also data at the top of the purchase funnel that might be even more germane to driving long-term sales.
Analytics should not be considered distinct from marketing teams. And as a part of that, marketing managers need to be comfortable with analytics empowering their decisions, rather than relying on someone else to tell them what to do.
The second part of this quote is not an analytics problem, it is a marketer problem. The marketer in this example has no idea how to use data, nor apparently any concept of the customer journey, and that is just not going to be a tenable situation in the modern economy.
Understand how algorithms and data map to business problems. Companies will see more effective data analytics if teams are clear on firm objectives, informed of the strategy, sensitive to organizational structure, and exposed to customers. To enable this understanding, data analysts should spend physical time outside of data analytics, perhaps visiting customers to give them an understanding of market requirements, attending market planning meetings to better appreciate the company’s goals, and helping to ensure data (IT), data analytics, and marketing are all aligned.
Again, this article is by no means all bad. This part on companies needing to focus on cross-departmental alignment for company-wide goals is 100% correct. The most successful companies I’ve worked with have every team moving towards the same north star, even if their tasks are different.
Why should one use a complicated model to present information when a simple infographic would suffice?
But then I got to this sentence.
Remember that earlier the authors felt the need to point out two things:
1. that it was very hard for companies to hire good analysts, a job description with now apparently includes the ability to design infographics
2. that it was a failure of analytics that a marketer might not know that there is data on top-of-funnel activity
So in order for it to be true that marketing analytics has failed to provide results, we have to live in a world where marketers need their hands held by analysts as they can’t be bothered to learn about numbers. Meanwhile, an analyst needs to be able to design a tracking strategy that accomplishes business goals, be a machine learning and big data expert who can pull learnings out millions of data points captured on a single customer, and they should have the design skills necessary to create infographics that can tell a story without words.
Look, analytics require work. The numbers won’t simply tell you an answer. But that was never their purpose in the first place.
Analytics and reporting enable business leaders to answer questions in a more thorough and complete manner than they were doing before. Good analytics are empowering, not prescriptive. They are not a replacement for your business sense, and should not be relied on to replace human thought.
If this is something that you are worried about for you own projects, then take a step back and evaluate what a success analytics program looks like. You can start by going through this list of 5 questions to ask when working with digital analytics data.
Get more articles like this directly to your inbox by signing up for the Proper Tack newsletter.
