Turn data-driven anxiety into data-driven confidence

A practical guide to cut through the noise to find the signal.

4 min readOct 6, 2020

I see many smart people get overwhelmed and lost when a number shows up the way it shouldn’t. It could be anything: sales, e-commerce conversion, or app performance. Data, when used properly, helps you to take action, set goals, and measure progress. But it can also become messy and create anxiety.

5 simple questions to reduce stress and bring alignment

Writing down answers to these five simple questions, whether individually or with a team, brings clarity and gets everyone on the same page. Writing it down may feel basic, but I’ve found that it reduces scrambling. It will save you days of wasted effort, bring you sanity, increase your confidence, and help you learn.

What do I see?
What should it be?
Why now?
What events happened?
What can we do?

These are useful in both technology and business. You can use these questions to triage a major outage or to create a roadmap of product features.

Where it goes wrong

Let’s say you just got a report that your conversation rate, or sales, dropped by 20% (you pick the metric).

A number of things can happen:

Your mind is flooded with questions. Is the issue technical? Is it a real or a blip?
50 people get involved and ask more questions.
A “war room” or “task force” is created by management.
Excel sheets are flying around faster than the two-minute video on how to create a pivot table.
The same questions are repeated, over and over.
It’s dark outside, and you haven’t left your desk in hours.

A path to clarity

Whether you’re organizing your mind or the minds of others, start by writing down what you see along the way. You can edit as you learn.

Question 1: What do I see?

Write down what you saw — this is the start of defining the problem.

Include the following:

Screenshot of the report
Time period
Filters

It’s ok if it changes over time. Get the first draft written down and only share it with key people.

Question 2: What should it be?

Write down what you expect. It should be precise, unlike “higher.”

Is it a goal or an expectation? Is your data seasonal?

Do you believe the data? Is this decline or increase real?

Question 3. Why now?

Spell out if it’s important and urgent.

How big of an issue is this compared to other work?

If it’s not urgent or important, then stop here.

Question 4: What events took place?

Anything you can think of that happened during the time period.

It doesn’t matter if it’s related or not. You can figure that out later.

Events include changes in code, personnel, monitoring, features, and holidays.

Question 5: What can we do?

Create a list of ideas that verify or change the metric.

List out everything you can do to change the metric. Include triaging that verifies the report is real. These become your hypotheses.

The word hypothesis may sound high brow, but it’s easy to create them by changing questions into statements. Hypotheses are statements that are either true or false.

For example, the question, “are sales down nationally, or is it regional?” becomes “Sales are down nationally and not regionally.” The statement is either true or false. It can be tested. It’s a hypothesis.

Your list should include anything to help you be confident in the data. It should also include any recent changes. For example, that 3 am code deploy.

Prioritize and take action

Once you have a big list of hypotheses, it’s time to prioritize and take action

Order them with the most important on the top. Move curiosities to the appendix.

Write down the amount of time it takes to get an answer to each hypothesis. It may not be worth testing if it takes a month.

Most importantly, write down how the hypothesis will change the metric from the first question. Pick the ones that have a meaningful impact and remove ones meant just for your curiosity.

If you have a team working with you, assign each hypothesis to an owner.

Periodically Review

It’s important to review your document daily or weekly, depending on the urgency. It becomes a record of your results and decisions. Review six or twelve months from now to continue learning.

This is the scientific method, and you didn’t realize it

These questions are based on the scientific method. You start with observations, construct hypotheses, test them, and learn. Enjoy!