How the Social Sciences Bookshelf is Transforming Consumer Insights

By Udaya Patnaik, Jump Associates

Over the past two decades smart folks like Daniel Kahnemann, the Freakonomics guys, Malcolm Gladwell and Michael Lewis have popularized the wonders of social and cultural research, behavioral economics, psychology and sociology. They’ve digested research and explained things that people often felt but didn’t know, like why having too many varieties of mustard in a grocery store might mean customers don’t buy any mustard at all. They’ve deconstructed and chronicled how fads like Hush Puppies shoes spread like wildfire. Dan Gilbert showed up in Prudential commercials to visualize optimism bias in retirement planning. Richard Thaler made a cameo in The Big Short to explain collateral debt obligations using blackjack and the “hot hand” theory.

Marketing and consumer insights professionals have been a willing and receptive audience of these ideas. That’s not surprising. These books, articles and presentations compellingly describe how and why people think and act the way they do and have clarified and legitimized the power of great insights. But insights professionals aren’t the only ones reading this stuff. People throughout corporations — specifically the leaders and teams that insights teams work with — eagerly devour these tomes and snippets searching for their own answers. In doing so, their expectations of what they want from their insights partners have changed.

Corporate leaders and teams…

Want more insightful insights from research projects.

The glut of easily accessible social science insights has raised the bar on quality for everyone else. Internal teams and their research vendors need to dig deeper to get to real reframes and unlocks, not just rehashed versions of what everyone read last week in James Surowiecki’s New Yorker column. Frameworks like the habit loop Charles Duhigg wrote about in The Power of Habit become important foundations on which to build, not the end insights themselves. That means insights professionals sharpening their data analysis skills to find patterns in people’s stories, possessions, attitudes and motivations and knowing how to coach each other to get to sublime findings. One large multinational food client recently had outside experts assess the quality of their insights work, only to find that less than half of what were labeled as “insights” were actually insightful. They’ve now embarked on a program to improve the quality of their research projects.

Consider themselves aware and even savvy about behavioral theory.

Product developers and finance leaders are more conversant about social science concepts because they’re discussed more frequently and found in popular culture. Nowadays someone can talk about semiotics in an advertising strategy discussion and expect to get head nodding around the table. The proliferation of TEDx events and videos, blogs and podcasts featuring the latest thinking allows everyone to have ready access to the latest ideas. Companies like Warby Parker and TriNet actually have formal book club programs for all employees in which social science literature enjoys healthy appeal. While that can make folks more open to seeing the value of deconstructing the whys of consumer behavior and perhaps less dismissive of insights, it has also turned a fair number of avid readers into know-it-alls, all too willing to question every finding or advocate for what they’ve read.

Are more skeptical than ever of static, data-only models.

Big Data is a major investment area for most corporations. The Moneyball phenomena spawned data-driven decision making in everything from sports to healthcare to farming. But human judgment still plays a large role. David Romer’s research has proven that football teams on fourth down needing four yards or less to get a first down should go for it — but most teams still punt. The recent Brexit vote and US presidential election reminded everyone of the limits of sampling, polling and number crunching to predict outcomes. Insights professionals that offer nuanced interpretation, empathy, consideration of psychological motives and qualitative narratives can show how predictions or estimates need to shift and when. Google Flu Trends famously whiffed in predicting outbreaks mostly because they didn’t tune their algorithm to consider the context of searches people were doing. It shut down in 2015. Of course, it’s a double-edged sword. With such a growing affinity for numbers, insights folks are also expected to provide more detailed quantitative data to back up their stories and interpretations.

Seek bold opinions and strategy not both sides of the story.

It seems like for every theory there’s a counter-theory. After The Paradox of Choice made the case for the too-many-choices problem other economists wrote articles saying that people also have “single option aversion”. Too many research projects — even ones specifically chartered to drive disruptive innovation — end in measured, non-controversial statements which drives leaders bonkers. It’s reminiscent of the old Harry Truman quote: “Give me a one-handed economist. All my economists say ‘on one hand…’, then ‘but on the other…’” Yes, there are often different interpretations but that doesn’t give permission to hedge bets, equivocate or make business partners make the call. Insights professionals have to be on the hook to get the facts, do rigorous analysis, validate learnings and make opinioned recommendations for what to do and not do. In one large CPG client, the veteran insights staffers are trusted advisors to the category general managers. They use feet-on-the-street empathy and deep category expertise to curate and create market launches and are renowned for their bright line clarity in decision making.

Given this backdrop, what should the savvy insights professional do? A few recommendations:

- Audit your insights work. Did you actually get to meaningful (if not mind-blowing) learnings from your last few research projects? If not, do you need to change partners or methods or raise the standards for what you consider insightful?

- Set up shared knowledge bases. What articles, books, videos, audio programs or blogs should everyone in the business be learning from to know the state-of-the-art in research? How can you disseminate learnings from projects and build widespread empathy?

- Provide context for data. What data gives a perspective that should be evened out with the right quantitative and qualitative information about people, motivations, cultural shifts and market variability? How do established models need to shift?

- Offer strategic guidance. Are you taking a bold enough stand with your recommendation that allows the business to make clear choices and tradeoffs? What clarity can you provide now and what questions can you answer later?

Social scientists and writers will continue to find, create, distill, inform and popularize theories about how we think and why. As that body of knowledge raises expectations of insights work it’s up to insights leaders and teams to keep upping their own games: pushing for deeper insights, sharing knowledge, giving a human lens to the data and providing stronger strategic recommendations. If they succeed their companies have significantly better odds of success. And there’s even a chance that the next book gets written about them.