Tips for protecting yourself — and your customers — from analysis paralysis
Understanding customers is essential. My worst fear as a Product Manager is shipping a thing that no one wants or needs to use.
Most people have been guilty, at one moment or another, of making up imaginary customer needs or extrapolating on what they know to reach a conclusion that isn’t correct. I’ve seen PMs, Engineers, Designers, Customer Service, and Salespeople at all levels doing this — myself included at times. It’s easy and tempting to do.
But is it possible to go in the other direction, to do too much research? Could research ever be a barrier to helping your customers? In my experience, it absolutely can.
Great PMs talk to customers all the time to understand their needs, but there has to be a balance: piles of research isn’t a substitute for generating customer value.
Not correctly defining your research goals can lead you down some serious rabbit holes, put you into “analysis paralysis” rather than “test and iterate” delivery mode and — worst of all — frustrate your customers.
This post will arm you with some tips on how to make your research meaningful:
- Define a hypothesis that moves the project forward
- Design research to get the data you need to make decisions
- Get demonstrable value from the time you spend with customers
Throughout this post, I’ll use the following scenario to illustrate concepts.
You work for a grocery delivery platform. You’re tasked with figuring out how to flag certain items as unavailable when they sell out to save your end-users and shoppers (who pick up and deliver the products to the end-users)time and frustration. You know that end-users and shoppers want this data. The question is, how can you get it? Your manager has asked you to do some research and make a presentation with your recommendation to the VP of Product in two weeks.
Define a hypothesis that moves the project forward
The best research helps business leaders make decisions in a data-driven way. Your objective should be to get data that will lead to a definite conclusion that will propel you to the next step.
A vast and common pitfall here is too loosely defining the decision at hand, which makes data requirements too broad. Maybe you’re not honing in on the right goal. You might be trying to solve a problem looking five years ahead, rather than focusing on market needs and adoption to serve your customers right now.
How do you narrow things down for your objective? Walk into your research with a hypothesis that, once you determine if it’s true or not, it will help the team decide on what to do next.
To get started, you need to decide what question you want to answer. Let’s go back to our scenario:
What you really need to know is: how do we get inventory data into our platform in the most timely and reliable way?
For the hypothesis, you need to turn this question into a statement that you believe to be true, something like:
[action/incentive/technology/segment/other] will get inventory data into our platform in the most timely and reliable way.
You can derive your hypothesis from a variety of sources, including:
- Previous customer interviews
- Input from customer-facing colleagues (sales, support, client services, etc.)
- Secondary research on the industry, customer segment, competitors or problem space
- Quantitative data analysis
Going back to our scenario -
After going over some transcripts from previous customer interviews, talking to colleagues and looking at industry data, you identify three potential sources of real-time inventory information:
1) Store managers who update orders to suppliers weekly and walk the floor
2) Shoppers in the store/who have been to the store in the last hour
3) Point of Sale systems
Based on your findings, you determine that Shoppers and Point of Sale systems aren’t reliably available and don’t tell the whole story about what’s really available to buy, so you write the following hypothesis:
Store managers will get inventory data into our platform in the most timely and reliable way.
To meet the requirements of the assignment, which includes a proposal on what to build, you decide that you think a new store manager feature in your company’s mobile app would easiest data entry. So you may add to your hypothesis….
Store managers, using a new feature in our app, will get inventory data into our platform in the most timely and reliable way.
Let’s pause to do a quick check on the scenario. The hypothesis is clear; the only caveat is that the most timely and reliable way is going to be hard to test in two weeks. We can modify it to something that will be more clearly relatable to our goal and reflects the shorter timeline for the project.
Store managers, using a new feature in our app, will get inventory data into our platform in a timely and reliable way.
Now that we have a hypothesis let’s look at the next tip.
Design research to get the data you need to make decisions
Even though we’ve narrowed down a hypothesis, there are still a lot of possibilities about how we could approach this project and the types of data we could collect.
I’ve observed this part of the process going sideways when the PM is not thinking about the desired results — the data you need to put into your deck or report to convince people that your idea will work.
What specific data points those are will depend somewhat on norms and practices in your company. If you can, ask decision-makers or trusted colleagues about what data they would need to see to endorse your hypothesis as a gut check.
Regardless of the amount of information you can gather beforehand, you know that the goal is to prove or disprove your hypothesis, this isn’t a general fact-finding mission.
What’s worked best for me as a first step, is picking 2–5 questions that I need to answer to validate or invalidate my hypothesis. Going back to the scenario -
You do some thinking and consultation and decide that there are three questions you need to answer:
1) Do store managers typically monitor inventory by checking the shelves/stockroom throughout the day?
2) Do store managers consistently carry a mobile device with them on the job, or have one accessible to them for work purposes?
3) Is tracking inventory for our app important enough to store managers that they are likely to complete the tracking consistently?
My next step is thinking about what type of data points I’d want to show to answer the question for my audience, and what an acceptable cutoff would be for supporting the validity of my hypothesis.
Since I worked in marketing consulting, I got into the habit of roughing out the actual slides I wanted to present for each question, minus data. This process gave me a good idea of the story I wanted to tell early enough in the process to make adjustments if it wasn’t flowing. Even if the data didn’t ultimately point to the direction of my hypothesis, the storyline was easy to adapt to a “this hypothesis is wrong, but here’s what I think might be right” story.
Let’s go back to our scenario and summarize what data points we are looking for, and our thresholds for supporting the hypothesis. Keep in mind my thresholds are arbitrary, and only there to give you an idea of how to structure the data you need.
1) Do store managers typically monitor inventory by checking the shelves/stockroom throughout the day? → at least 75% of store managers in the study group do a walkthrough of the store to check inventory 3x or more per 8 hour shift
2) Do store managers consistently carry a mobile device with them on the job, or have one accessible to them for work purposes? → 85% of the store managers identified question #1 always have a mobile device used for work with them during their shifts
3) Is tracking inventory for our app important enough to store managers that they are likely to do it consistently? → in stores where our platform accounts for > 20% of their revenue, 85% of store managers say they would update inventory in a mobile app at least 2x per shift
So, if all of these statements are true, the hypothesis is valid, and we should continue building an in-app feature for store managers to track inventory. If one of them is false, the hypothesis is invalid, and we should look into the other two options identified in the hypothesis generation stage (Shopper reports and/or Point of Sale data).
To close this out, let’s go to the third tip for staying out of research purgatory.
Get demonstrable value from the time you spend with customers
Taking a customer’s time is entering into a social contract with them. In exchange for answering your questions, you create an expectation that you’ll get back to them with some kind of response — ideally a result that helps them — in a defined period.
Talking to customers is excellent, but it should always happen with a healthy degree of caution about the expectations you’re setting with them.
The good thing about caution is it prompts us to prioritize — you need to maximize the value you’re getting from the time you’re requesting. With the questions above, you have a great start. Now you need to sort out how to most efficiently get that data from them and honor your social contract. You can do that by answering the following questions:
- Who exactly do I need to talk to, and why them?
- What time commitment do I need from them?
- Do I have specific questions that line up with the time commitment?
- What format should the interview be in to get accurate data with minimal time commitment (phone interview, email, survey, onsite observation…)?
- When will I be able to share an outcome with them?
This forces you to think through your ask and ensure it’s reasonable, and that you’re laser-focused on what you need to find out. Additional information might warrant follow up later, but honing in on these answers will help you stay focused on your research goal and positioned to gain customer trust as well as participation.
In our scenario, with well-defined objectives and well thought out goals, you are also more likely to get a decision about the next steps from the VP of Product, which you can then share with your customers.
Putting it together: Avoiding analysis paralysis
As an ex-professional researcher who just likes knowing stuff, being disciplined in my research was tough when I started. However, it’s paid dividends in helping me sell ideas and tell compelling, data-driven stories to get organizational buy-in and features built that we could offer customers.
Excellent PM research doesn’t need to boil the ocean, involve hundreds of customers, be the last word on the topic, or change the course of the company. But it does need to move you forward and get results for customers.
I hope these tips are useful to you in your next project — please leave your questions and feedback in the comments.