3 Decisions Startups Should Make to Maximize Learnings From User Validation Surveys

Building a survey is a widespread method to get a first glimpse of how potential users might respond to your future product. It is also pretty easy, quick, and cheap since all you have to do is come up with a bunch of questions, write them down in a google form (or any other form), and send it out to a large enough user group that fits your audience.

While this method is simple to execute, it often requires careful planning of the questions to include for it to be effective. Since you are most likely planning to rely on the survey’s results to validate your product direction, it is crucial to make some preliminary decisions to avoid pitfalls that can make its results unreliable or unusable.

Let’s start by understanding what validation surveys are and when we use them:

While there are different surveys you can send at various stages of the idea development process, in this article, I will be referring specifically to surveys you send at the beginning of the ideation process. At this stage, your product doesn’t exist yet, and you have only identified a possible pain point that you believe you can solve.

For example, you found that many business owners complain that the process of sending tax documents to their accountants at the end of each year is too complicated. You consider building a digital solution that will allow business owners to transfer documents to their accountants quickly, and you plan to sell it to CPA firms as a white label (a product that is produced by one company but is sold under another company’s brand).

Before you spend time and money building and marketing your solution, you send out a survey in groups of accountants and business owners, asking them questions to get your first insights to validate how you should move forward with your product idea.

To summarize, your goal with this survey is to decide whether you should move forward with your idea, change direction, or drop it completely.

How to make sure your survey results allow you to make the most educated decision?

After years of building validation surveys, I gathered the 3 most important decisions you should make before you start building your own survey. Making those decisions will help you learn more from your survey results and save time for you and your survey respondents.

#1
How many surveys do I need?
To make this decision, you must decide who your target audience is. Typically, your primary target audiences are those who will pay for the product and those who will use your product. To validate your audience choice, ask yourself if the answers of this specific group of people will impact your decision on how to move forward. This target audience will only be relevant for your case if they do.

Let’s look back at the CPA firms’ example. We can see that there are 2 possible target audiences:
1. Accountants — since they will be the key influencers on the CPA firms’ decision to buy your product, and
2. Business owners — since accountants will only buy the product from you if their clients have a better experience and they can increase their revenue by signing up more clients.

There may be a different number of primary target audiences for different ideas. For example, suppose your vision is to connect actors, producers, and directors. In that case, you will have 3 primary audiences, while if your idea is to build a meditation app for single women, you will have only one primary target audience.

It would be best to build a different survey for each target audience since your goals and intentions from asking them questions may be different. In the long run, the more target audiences your digital product covers, the more complicated the solution will be and the more money, resources, and effort you will have to invest in building it. Therefore, I recommend carefully choosing your focus.

#2
What do I want to learn from the survey results?

To avoid collecting unusable data, you need to know what you expect to learn from the answers to the survey questions. Consider the key factors in your decision to move forward or drop the topic. You may need to know how your audience currently deals with a specific situation, how critical is solving a particular issue for them, or if they have the required time and resources to use a solution like yours.

In the CPA firms’ example, you would probably like to know if accountants will agree to consider implementing a new system in their process, how critical customer satisfaction from the annual tax process is for them, and how they currently deal with organizing the documents coming from their clients. Alternatively, you could learn from business owners if an annual tax-return app will be a factor in their choice of accountant and what value they believe they will get from it.

Spend time thinking about all the information you need to make an educated decision about your product direction. The more accurate you’ll be at this point, the more focused your questions will be, and the better results you will get.

#3
How am I planning to use the answers?

Last but not least is setting your intentions for the survey results. These intentions will be used as a compass when deciding what type of questions you want to include in your survey and how to phrase them. You will gain clarity about what you want to learn from the answers, and your questions will be more targeted and provide more value to your decision-making process.

Going back to the CPA firms’ example, you can decide to move forward with your current idea only if you find that accountants might agree to spend time implementing a new product to improve the annual-tax process. In this case, you should ask about their current process, the likelihood of them changing it, and how easy it will be for them to change it.

If their current process is simple, and they are not likely to change it, you might want to find a different solution to solve this pain point for business owners or drop the idea altogether. However, if you find that accountants don’t currently have a sound system for collecting tax documents and that they will gladly invest in improving this process — you might decide to move forward with your idea.

Choose how to use the survey results carefully. Make sure that what you expect to learn from each question is indeed what you will learn from it. For example, a typical mistake is to expect that if you include characteristics like location, age, or sex in your survey, you will learn who your users might be and what they might like. Unfortunately, that’s a false expectation.

Since you are not about to spend thousands of dollars on market research at this stage, you will most likely distribute those surveys on social media, events, in your circles, or any other low-cost (or free) solution. Those audiences already have some bias towards at least one of the characteristics (age, location, interests). In addition, the number of answers from these surveys is usually tiny, and the results you will get cannot conclusively detect what might characterize your overall target audience.

It does not mean you shouldn’t include these questions in your survey to get a sense of where in life the person who answered the questions might be. However, ensure you are using them correctly since you want to avoid building a product on top of a false assumption.

What comes after you make the decisions?

After having a clear plan of how many surveys you need to build, what you want to know from each target audience, and how you will use the survey results to move forward — it is time to build the survey. In my next article(s), I will dive deeper into the most efficient survey structure and the most relevant questions you should (and shouldn’t) ask. Feel free to follow me and get notified when they are published.