In the Age of Data: The New Number You Need to Grow
In 2003 Frederick F. Reichheld posted an article called “The One Number You Need to Grow“. In it, he presented the NPS (Net Promoter Score), a simple but powerful metric that is used as an indicator of customer loyalty. Since then, this metric became the standard not only for loyalty but also for success capability, because the result had a strong correlation with the ability of a company to succeed. Using the NPS as an inspiration, there is a new indicator with a similar impact on a business that has proven difficult to measure: How effectively a company uses the data at its disposal to make better decisions. In this article, we will propose an approach to deal with this topic and highlight the importance of this measurement.
The effective use of data
In MercadoLibre, data is used extensively in our products and services not just as an operational asset but as a key ingredient for developing better ones. In terms of strategy, data has changed the way we make decisions. To shape MercadoLibre as a data-driven company we have developed an analytics strategy, in which the data, the technology, and the skills can thrive to generate value for the company.
So, having everything we need in place, we wanted to know how well our implementation was going. In other words, how data-driven we were.
The dependency trap
In our research, we saw that one way used by consulting firms to assess if a company was data-driven or not was by means of a checklist of the recipe: data, technology, and people. But in our experience that qualitative check does not guarantee a true impact of the data in the business. As we’ve seen in this article on data literacy, having just one department in the organization developing super-advanced models while the rest of the organization continues making gut decisions does not turn it into a data-driven company.
Other approaches
Another approach is to try to answer a long list of specific questions; here are a few examples:
- % of data cataloged per business domain
- % of data in legacy technologies
- Number of users per data tool
- % of the revenue that comes from data product
- Total number of hours in data training
Each metric is useful indeed but does not necessarily answer the question of how data-driven the company is. For instance, we can invest a lot in training but that does not translate directly into a more data-savvy behavior.
Yet another approach is to rely on those statistics that try to monetize the value of data: “Our recommendation engine accounts for n% of the revenue”; so the rest of the revenue is not data-driven? These measures fall short because it is almost impossible to differentiate what is data-driven and what is not.
Lastly, other approaches are based on multiple metrics (like the questions we’ve asked before) with specific weights that create a single metric. We’ve ruled them out since the calculation was too subjective and complex. We feel that while it is obvious to follow different metrics given that different components create what we call data-driven, in the end, the only important metric is if data generates value for individuals.
So until this point, the existing approaches to measure “how data-driven we are” didn’t prove to be very effective. Thus, we decided to create a measure we called “The Data-Driven Index”.
Data-Driven Index
Given that we have the data logs of use for each data tool by all users, we thought we could create a measure based on that, assuming that no professional would spend their time concurrently on an activity that does not generate value. In other words, a senior executive who regularly consults a dashboard is searching for a guide for making informed decisions. or else he would probably stop consulting it. An analyst executing database queries is using that information to enrich some activity in his role or he simply would not continue.
Under this assumption, it is possible to resort to the information on the uses of different data tools to size the value of the data for each user and consequently, for Mercado Libre.
The calculation of the indicator is divided into two:
1 A Data-driven individual finds in data a way to generate value; so does the one who frequently interacts with or creates any data resource like dashboards, queries, databases, notebooks, or models. They do so with certain recurrence and use various types of tools depending on their application. Conversely, a gut-driven individual probably has little direct interaction with any data source and, if ever consulted, makes limited use of it.
2 A data-driven team consists of a large enough number of data-driven individuals who can influence team decisions with data. We call it a data-driven team when all members share the same approach to data or when the data-driven individuals outweigh by number or expertise the gut-driven individuals.
Calculation
- Data-driven Users
For user segmentation, we used a methodology called RFM, which stands for Recency, Frequency, and Monetary value. This segmentation is useful because it describes the different behaviors to foster greater engagement. Its main advantages are:
- It uses objective numerical scales that produce a concise and informative high-level user description.
- It’s simple, quick, and flexible in its application.
- It is intuitive: the result of this segmentation method is easy to understand and interpret.
The variables we used were frequency, recency, number of tools used, and also the number of creation tasks by the user. Each variable was pre-processed in many different ways but the most important point was probably to leave it as a relative percentile for the user’s role. Given the nature of each role, managers’ usage was compared with that of other managers, analysts’ with other analysts’ and so on.
The result was five segments of users, from the ones with the most sophisticated use of data (Champions) to those who used data for more rudimentary tasks (Basic).
2. Data-Driven Teams
Now that we have calculated how data-driven each individual is, we can measure the whole company or any team inside it. To calculate the Data-Driven Index (DDI), the following formula is applied:
The principle of this formula is measuring what proportion of people in a team are data users, but more specifically, if the team has a lot of expertise (proportion of champions over basic users) in which case the power of data will be more extended across all the team. Users with a lot of expertise on data will make up for the lack of or limited expertise of other users. However, if a team has only basic users, the use of data may be just operational, so probably it is difficult for that team to be data-driven.
In short, this gives a team two options for teams to become data-driven, either to increase the overall use of data tools by a greater part of the team or to increase its expertise in order to compensate for the part of the team with no data access.
Conclusion
What you cannot measure, you cannot improve, so having a data-driven index is a game-changer to monitor and create actions to improve the indicator and become an even more data-driven company.
As you can see, the calculation is simple, the interpretation is straightforward and the classification of each user gives a clue regarding the actions that can be taken to increase the data-driven index of a team. For us as a company, having a data-driven index allows us to measure in a single metric how effective the data-driven culture has been. It shows us where inside the company we have to be more focused on and how different strategies have an effect on this metric and its users.
Still, there is a lot of room for simplicity in this measure. Does your company have a similar indicator? Can you think of a better way to measure how data-driven a team/company is?
We’d like to learn about your approaches and solutions!
