Your Latest Consumer Behavior Report Is Late. There Is a Fix.
If you are reading the latest consumer food basket report, I bet you are late. Why ? The consumer behavior in 2021 is not the same as 2018 or 2019. This divergence can still be explained as the time scale is in years. But in category like food, two consecutive week might be surprisingly different, telling us that characteristic of the consumer’s choices has changed in this short time frame. Shouldn’t your communication change too ? Company’s actions could become more adaptive by mapping it to consumer’s behavior at any point of time.
Here, I use food basket as an analogy, but it can just easily studied for any other consumer preferences. If your business is to know consumer sentiments on food or the latest food basket composition than reliance on survey based report is fraught with risk, risk of redundancy.
But why is it late ?
Most of the data used to analyze consumer sentiments, preferences, choices are made using the traditional survey. Often these surveys are conducted physically with survey team standing outside kirana, retail stores — large and small, recording consumer purchases and asking them questions on their preferences. The data is then aggregated for the the purpose of discovering the consumer’s insights which is consumed as a report by the industry.
The consumer survey were designed in the pre-internet years. In these surveys, one would ask a set of people questions on their choices and preferences, with a primary objective to build collective consumer insight. The survey team would be spread across geographies. The process was time consuming and expensive. For a country as large and diverse as India, these would take weeks, and sometime, a month — collecting all the physical copy of surveys in one place, manually reading through the forms, and then doing the analysis. The diversity of data also makes the insight less reliable because for each unique geography your sample size is limited.
A report which claimed to have found insights on India’s consumer behavior during the pandemic
I don’t come from the marketing world and never felt the need to critically think about survey until I saw a headline grabbing report. Published by a mainstream and well respected consumer research company an insight report on how India and rest of Asia had changed during the pandemic of 2020. Though everyone in the business would know about this research firm, I didn’t recognize the name. Much later, a friend, veteran from the industry told me about the brand name change. But when I read the report summary in business newspaper, my curiosity was piqued as I wondered if they have found a way into consumer mind, some artificial intelligence breakthrough. So I browsed through the report. I was expecting a new research on extracting ‘consumer thinking’ from some data (which I had overlooked) as I have also been researching a way to build consumer profile from alternative data — this explains my out-of-ordinary expectation. I wanted to know more about the method and even more — what data have they used. The final page of the report in a way relieved me in some odd way. All the ‘consumer thinking’ came from the survey.
Why we should give importance to number of participant ?
But I couldn’t help notice the finer detail. This claimed understanding of Indian consumers came from some 1545 surveys conducted in India. Maybe this is completely reasonable size datasets for some hard to get research subject but if I am a brand owner, I will realize that this number doesn’t even come close to mean. The variation from the actual mean would high enough to make this data inconclusive, leave alone any insights.
India has more than 36 states and some 634 districts. At some level, each district will behave different than a district 200 kms away in a given time-frame. West Bengal will eat different than Orissa in a particular week. But there is nothing which proves that this choice relationship is linear based on the proximity, that Bengal’s consumer choice will be different from Tamil Nadu just because they are further away. Now things becomes interesting as you add new dimensions — rural/urban, male/female, age groups. One will not be too stressed to question if 1545 samples size can account for these variation in consumer set. These samples couldn’t have possibly approximated India. Yet, we rely on this.
Can we have a better approximation of consumer preferences, habits and behaviors from data other than survey.
An alternative source. The analysis will be as hot as a bun from a baker’s oven and as reliable as law of average entails (but with millions rather than hundred of people’s answers)
Google Search is one such alternative. Today with phone as ubiquitous as a tooth brush, everyone has one for himself. One can make an accurate guess that when someone is interested in something — he will use his phone and search the internet. More often than not, this query of his lands inside google search. And one needs to mine this information and build a knowledge from it. Importantly, unlike limited number of participants in a survey, search results by google looks at millions of searches in India.
Its challenging but doable. You need some imagination and bit of data science. The advances in artificial intelligence with new state of the art natural language models like Bert, GPT-2 make this possible too. Using natural language model to find insight we will leave for another blog post.
You can get amazing insight if you know where to look. For consumer insights, we haven’t yet scratched the surface with alternative data — social media being one of them. Lately, I have spent considerable time looking at Google’s dataset, beyond the google trend which most of us hear about. From one of the analysis — I was able to unravel the ‘rise from the ashes’ moment for the juice segment from this data. I didn’t have any privilege sales data from the juice companies, nor did I ask people about their purchases. I made inference from their actions, which this data represented. Sooner than later, an article appeared in the business paper with news that juice was doing well. Deeply satisfying to know that signal like this can be learnt in advance. You knew this months ago, while it was happening.
Sure, if one wants opinion on a narrow and specific subject, one ask in the form of survey, digital or otherwise. But I would first try to get answer without asking. Then, I would avoid lot of bias (hbr: sign-in might be required)
The technology of today allows you to know what a country ate yesterday, eating today, will eat next week. Using the data out there, brand can make their decision making more scientific and robust. Prediction is more accurate and planning more certain. So what does a food brands with knowledge of consumer eating habit and weekly behavior do ? Sure, you profit from it, by optimizing your marketing and sales efforts. You choose to reduce your marketing budget or make it more effective with communication adapting from week to week, location to location. But I want you to imagine beyond this — remove some human gut feeling and rely more on the data for your decision making whether small or big.
