Why Asking The Right Questions is the Underrated Data Superpower
3 Tips to Help You Ask The Right Questions for Your Data Projects
Muturi Njeri
In December 2019 LinkedIn released their annual report of the most in-demand professional skills. To little surprise, data science-related skills dominated the list: artificial intelligence, analytical reasoning, robotics engineering, business analysis. Seven years earlier, the Harvard Business Review had declared data science the “sexiest job of the 21-st century.” Yet, the meaning of data science itself remains contested. As I was learning my basics in data science, I found Doug Rose’s definition insightful. Rose posits that data science is the process of applying the scientific method to data to derive valuable insights from the data. Coming from a social science background, I found his emphasis on not just the data but also the empirical process — asking questions, formulating hypotheses, obtaining data and running experiments to test the hypotheses — powerful. His definition places inquiry at the centre of the process: shining the spotlight on the “science” in data science.
“If I had only one hour to save the world, I would spend fifty-five minutes defining the questions, and only five minutes finding the answers.”- Albert Einstein
Many organizations are heavily investing in the expensive infrastructure, software and professionals needed to “do data science”, but fewer are investing in building a culture that constantly flexes the muscle needed to ask the right questions that kick start the process. In some organizational cultures, any type of questioning is perceived as an unnecessary bump on the road to action, if not as an affront to authority. There is an established order; questions threaten that order. Yet, the path to the insights that lead to innovation is lined with questions. What do our clients really need? Why are a lot of them complaining about our services on social media? How can we reduce our cost of production? How can we influence our customers to buy more frequently?
Stefaan Verhulst, founder of the GovLab, an action research centre in New York, points out how our instinct to jump to answers before understanding the important questions is a missed opportunity at a time when the abundance of data could potentially help us solve the grand challenges of our time. Yet data itself won’t solve the problems, especially if we don’t ask the right questions. But if we ask the right questions, we avoid a common pitfall: making datasets the hammer we keep finding nails for.
For data science teams, the quality of the questions they ask — and how they refine them in relation to their understanding of the business challenges and data available — is doubly crucial. Yet even data scientists sometimes rush to the more technical parts of the process — analysis, modeling, testing, visualization — without paying close attention to the question (or problem) their models need to solve.
Admond Lee, a seasoned data scientist and data communicator, tells the story of how at his first data science internship at CERN (the European Organization for Nuclear Research) he was so excited to “get his hands dirty” that he paid little attention to the bigger picture of what his analysis was meant to achieve. It took him over two weeks of trying to clean the data to realize how his lack of understanding of the problem and data constrained him. Luckily he had time to fix his error but for many data teams, failure to fully grasp the business problems and thus refine their questions — and subsequent work — can lead to models and insights that, while technically interesting, are difficult to deploy or yield insignificant ROI for the business.
***
While asking questions is easy (otherwise my 3-year old niece wouldn’t be asking three every two minutes), asking the right questions is tough. Here are three tips to help you generate and refine the right questions, especially when working on data projects.
1. Ask questions that go after the ‘bigger picture’
For data science teams to ask the questions that address the core needs of the business, they must first understand the business context — and the priorities that form the key themes of the organization’s story. Often, data teams tend to develop powerful models to solve nitty-gritty problems that do not point directly to the organization’s strategy. While the details are crucial, given the wide range of possible questions, a wiser strategy is to prioritize the questions that go after the larger truths in the organization. For instance, an online retailer has noticed high drop-off rates between the time a customer starts a check-out process and completed orders. While it might be interesting to ask how hour-of-the-day correlates with order completion rate, the larger story here is figuring out why the customers are dropping off and what can be done to improve their experience so as to drive sales.
Sometimes, focusing on the smaller questions — which Rose aptly calls “looking through the keyhole of a glass door” — obscures the larger picture, the purpose of the analysis or experimentation. This usually occurs when you have a large number of analysts looking at different, small data points without clear guidance on the overarching question. As such, the data science leader needs to provide the context and frame the questions — as well as push other members of the data science team to go after the bigger questions which lead to valuable insights for the organization.
2. Assess the assumptions and evidence behind your questions and statements
In their seminal book Asking the Right Questions: A Guide to Critical Thinking, Browne and Keely discuss two approaches to thinking about information as we process it: a sponge approach and a panning-for-gold approach. A sponge thinker tries to absorb as much information as possible as it is presented while a panning-for-gold thinker actively evaluates the information — questioning the assumptions, logic and evidence as they go. While both approaches are useful in different contexts, most of us have had a lot more practice with the sponge approach, especially given that our learning systems typically reward memorization more than critical thinking. To ask interesting questions, we need to adopt a panning for gold approach. We need to unearth the “common sense” assumptions hidden in statements, evaluate the logic used in arguments and assess the quality of the evidence used in supporting certain positions.
In the online retailer example above, we might question the implicit assumption that changing, say, the number of steps in the check-out process would lead to more completed orders. This provides us with another hypothesis that we can experiment on to figure out if it is supported by actual customer behaviour.
3. Build a culture that encourages questioning and experimentation
Finally, to ask the right questions, you need to build a culture — within the data science team and across the organization — that encourages questioning, reasoned arguments and experimentation. In his HBR’s Ideacast podcast episode, Professor Stefan Thomke tells the story of a Bing employee who came up with a new idea on how to display ads on the search engine. His manager wasn’t receptive to the idea, but the employee ran a controlled test with the idea and found that ads did better in the new setup. Eventually, the idea generated over $100 million dollars in additional revenue that year alone. While the manager was initially unmoved, the employee felt empowered enough to ask questions about how to improve ad placement — as well as run experiments to validate his hypotheses with actual data.
Unfortunately, many organizations favor authority and intuition over experimentation. While this approach may work in the short term as leaders mine their experiences and influence to make decisions, it is risky in the long-term as such leaders transition from their roles. In our world of heightened VUCA (volatility, uncertainty, complexity and ambiguity), the risk is untenably high.
Acting on the insights drawn from data projects powerfully reinforces a culture of learning and experimentation. In some organizations, employees ask the right questions, gather the right data and run experiments but their insights are shelved. Nothing happens differently. This sends the wrong signals and discourages further experimentation. As a business leader, you can empower your employees by setting clear strategic goals as well as availing tools and training that enable them to ask and answer important questions with data. Then, using their answers to achieve the strategic goals of the business.
***
Ultimately, my challenge is two-sided: to data professionals and to leaders in organizations seeking to leverage insights from data. To data professionals: as you build technical skill-sets and sophisticated models, how are you establishing the right problems to solve? To leaders in organizations: as you invest in architecture and talent for your data projects, how are you building cultures and systems that put the right questions at the core of your data mission?
For more information about the Data Science Xcelerator program, please click here. We’d love to work with you to develop data skills for you and your team.
About the author: Muturi Njeri is a Data Science Design Consultant at Xcelerator. He is passionate about education, creative arts and African development. He loves to read and write on education, development, social justice, technology and the arts. You can read more thought pieces from him on his blog or follow him on LinkedIn.
