How to Deliver Real Data-Driven Insights
What is a real insight, how to uncover them, and the cost of not finding them
Co-written by João Sousa, Director of Growth at Kausa, and , Data Manager at Traveloka.
These days everyone is looking for insights. Read any job description the analytics teams’ mission is to “produce insights”. But this mission is very ambiguous. Firstly, is their role to uncover these insights (i.e., conduct deep analysis and communicate the insights) or just to enable the business to find them (i.e., build out dashboards that most times don’t deliver any real insights)?
Secondly, the definition of insight is quite vague and this term is often misused. Most people refer to observations as insights.
As a result, data practitioners are often frustrated that the business isn’t acting on the “insights generated”. In reality, they are not sharing real insights. This hinders teams from delivering real business value, and wastes both data and business teams’ time, making it very costly to go after so-called insights.
What is an insight?
An insight goes way beyond just finding something “interesting” in the data. Showing what is happening on dashboards is not sharing insights. Among the various descriptions out there, my favorite is the one from Brent Dykes, focusing on 3 criteria:
1) Shift in understanding: not just pointing out an observation or an irregularity, but also the reason that caused it
2) Unexpected reason: something that wasn’t intentional but surprising
3) Aligns with what you/stakeholders care about: closely aligned with business goals
And boiling it down to “an unexpected shift in the way we understand things that inspires us to act.”
Sadly even actionable insights can go wasted. So I’d like to add a 4th criterion:
4) Effectively communicated
Your audience can only act on something as long as they fully grasp it. Otherwise, the situation might remind one of the philosophical thought experiment “If a tree falls in a forest and no one is around to hear it, does it make a sound?”
Now that we have a clear end goal, how can you go beyond observations and deliver real insights?
How to uncover insights regularly
Actionable insights start with a why question, not what.
You won’t find any insights on dashboards or reports. They just monitor metrics and cover high-level hypotheses (i.e., the usual suspects). Thus you’re only testing recurring hypotheses that just scratch the surface. You need to leverage all available data instead of just looking into a small portion. (Check this article to learn how to break other bad analytics habits.)
The key lies in diagnostic analytics. Understanding why business metrics change and what drives them is the way to deliver real insights. Start by asking how the changes in your key business metric meet the 3 criteria defined:
- What’s causing the change in the business metric?
- Did we plan for this to happen?
- Is it tied to our business priorities?
Uncovering these real insights requires drilling down to the why — i.e., trying to find more granular insights leveraging all available data and combining multiple factors/dimensions and related metrics.
Status quo
Let’s break this down with an example. Assume you’re a data analyst at an eCommerce company, and for the past 3 weeks, the active user base (i.e., users who visit your app/website) hasn’t been growing. The growth/marketing department is concerned and has asked you to investigate further.
You know the metric (Active user base) and its trend (i.e., flat over the past 3 weeks). We hear that most teams would first check the data accuracy… what’s the fun in going through the trouble of digging through the data when you can blame the data quality?
Observations
Once you have confirmed the data is accurate, it’s time to put your thinking hat and come up with a set of hypotheses:
- Breakdown the numbers by country & region to compare trends
- Check seasonality — events like school holidays or even long weekends tend to have pull forward effect
- Perform cohort analysis (i.e., new vs existing users), based on acquisition month/channel, etc
After spending a considerable amount of time, you have figured out the number of new users (i.e., downloads) has dropped, which further contributed to active users remaining flat in the past weeks.
This is interesting, but not an insight. It only partially answers the question, and there isn’t any concrete action that the business team can take based on this. You need to dig deeper into a real insight.
Real insights
Upon brainstorming, you have identified a few additional hypotheses on why the new users could have been dropping. After many slices and dices, you zeroed in on a finding that conversions from a specific search term have dropped in the past few weeks. This is a really good finding, but still incomplete. To close the gap, you analyze specific search terms on the app/android store and found out that your competitor is running an ad campaign for this term. The competitor is taking away your installs.
YES — now you have figured out an insight that’s truly actionable for your marketing team. Now they can counter this campaign and bring back the app install numbers.
Implications of a reactive approach
You finally found the insight. However, it took a significant amount of time and you uncovered this 1 month after the new ad campaigns launched by the competitor. In the last month, you lost many active users, which had a significant impact on the business performance.
Maximizing the value of data requires proactivity. And not taking action till you observe a sharp decline in your dashboards, and then spending days on root-cause analysis can be very costly.
Ideally, the data team would not wait for the growth/marketing leader to ask why this is happening, but rather proactively share insights and recommendations. Within this scenario, the action could have been taken significantly earlier, cutting the losses to a minimum and taking quick action to turn the outcome into a positive business impact.
Why most teams are not uncovering real insights
The most data-advanced teams proactively track metrics and investigate changes to share insights and recommendations. But a lot of teams are still very much descriptive (i.e., what’s happening), only identifying high-level trends and overlooking diagnostic analytics. Many causes contribute to this situation, as I explored in the article “Diagnostic analytics gap”.
How to close this gap
Closing this gap requires proactive “full force diagnostic analytics” (see article for more detail). Where to focus depends on the starting point across each element (culture, people, tools, processes). In a nutshell:
- Culture: Promote a data culture of drilling down to the why over just staying with business-led high-level hypotheses (i.e., the typical answers — it’s seasonality, it’s because of a new marketing campaign). Ensure alignment between business and data stakeholders to drive initiatives with executive buy-in.
- People: Embed analysts into business teams to develop business and domain expertise. Promote a strong collaboration between data and business teams. Create a shared understanding of the most important use cases/metrics where analysts proactively share insights.
- Tools: Consider decision intelligence platforms to augment existing workflows to run comprehensive and fast analysis, enable teams to proactively understand metrics changes, and share data-driven insights and recommendations.
- Processes: Review current diagnostic analytics processes. Ensure that teams have consistent methodologies and approaches to perform root-cause analysis. Develop best practices in sharing insights and recommendations between data and business teams.
Bottom line:
Teams that uncover unexpected reasons that create a shift in understanding and ties to business outcomes do so by:
- Creating a strong collaboration between business and data teams
- Drilling down to the why by leveraging all available data, looking into multiple dimensions together, and taking related metrics into account
- Setting clear processes on how to distribute and act fast on these insights.
—
Thoughts? Reach out to João Sousa, Director of Growth at Kausa, or Umesh Ramakrishnan, Data Manager at Traveloka. We’d love to hear from you. Our objective is to share our collective experience to empower data teams to deliver more business impact.
Stay tuned for more articles on how to increase the business value of data & analytics.
Note from Umesh Ramakrishnan: Views expressed are my own and not a representation of my current employer (Traveloka)
—
References:
- Brent Dykes, Insight Literacy: Why We Need To Clarify What Insights Really Are (2022), Forbes.com
