How to conduct effective quantitative research
My name is Ilya Dyanov and I’m a CX project manager at inDrive. In this article, I will share the key rules for conducting effective quantitative research.
Published by Ilya Dyanov.
Let’s say that we’ve already come up with certain hypotheses, performed the necessary qualitative research, and are ready to move on to our quantitative study.
When designing a quantitative survey, we aim to:
- validate our hypotheses with numerical data;
- make the survey easy for users to understand;
- achieve a high conversion to responses.
How to conduct the perfect quantitative survey
- Determine your audience
Before designing a quantitative survey, you need to identify your target audience. These are the respondents who will be taking your survey.
When analyzing your results, it’s often necessary to isolate a segment of your audience whose answers you’re going to work with. For example, this could be users with specific experience with your product or service, or users who meet certain parameters.
Say, for instance, we’re only interested in respondents who have used the service or product over the past month.
Check if you can send a survey to the audience, initially segmenting it based on this criterion, so as not to add an unnecessary question: “Have you used our product or service in the last month?”.
2. Consider the design of your survey
Remember, a quantitative survey aims to solve the research problem using questions that have been clearly formulated and determined in advance.
In other words, the survey should only include questions with answers that will help you validate your hypotheses and identify areas of growth.
Questions that “would be good to know” should only be included if the sample size allows. After all, the more questions you include in your survey, the fewer responses you’ll receive, and your conversion rate will likely drop with each additional question.
There is one exception, however: the preliminary question, which guides the audience to your key questions or helps them understand the context of the survey.
Another common mistake is a pre-survey with a satisfaction rating of the quality of the service or product, known as CSAT.
This metric is only relevant if you plan to:
- compare certain parts of the service/product in terms of customer satisfaction;
- conduct the survey regularly to track metrics over time.
In all other cases, it’s best to avoid preliminary questions with quantification. There’s no point, and your conversion will drop.
3. Ensure that your survey is as simple and understandable as possible.
When conducting a survey, you want to avoid the so-called “déformation professionnelle” at all costs. Although a question might make sense to you and your colleagues, it could be interpreted completely differently by the users.
You can prevent this bias by putting yourself in the respondent’s shoes. Will they understand exactly what you’re trying to say?
4. Close-ended questions
And we’re not talking about simple yes/no questions, but rather, questions that offer the user several answer options.
Here, it’s especially important to randomize the answer order for each respondent. Basically, the list of options should be in a different order each time the survey is taken.
This way, you significantly increase the statistical significance of the results and eliminate the chance that users will just choose the most conveniently located answer.
If the survey tool you’re using allows you to add an “Other” option, where the user can enter their own text, be sure to include it.
You might think that a question is brilliantly formulated, but if you include an “Other” option, you’ll find that it’s rarely left empty. Give your respondents the chance to speak their minds!
5. Choose your survey tool wisely
When deciding which platform to use when conducting your survey, always consider the tool’s various features and capabilities.
There are a number of popular solutions on the market. However, some of them, for example, don’t record the user’s answers if they fail to complete the full survey. What does that mean?
If this happens, you won’t just lose the user’s answers to the last few questions; you might lose all of their answers entirely.
If your survey includes quite a few questions, make sure to double-check that every single question is critically important to your research at this time.
Even if your survey tool lets you record a user’s answers, whether or not they finish the survey, you should always put the most important questions at the start.
If you have a lot of questions, and the tool doesn’t record answers to unfinished surveys, try to split the survey up into several parts (surveys within a survey).
6. Enable UTM tags and post-analytics
Most survey tools allow you to add custom parameters to the URL of a survey link. These are known as UTM tags. Any time a user submits their answers, you’ll be able to record their unique identifier or any other parameter. For example, you can use UTM tags to determine a respondent’s city.
When designing your survey, you might not consider this information particularly useful. However, you’ll find that experience shows otherwise. In fact, these tags might save you from having to conduct additional research.
By performing a proper post-analysis of your quantitative survey results, you can significantly enhance their statistical significance. During the survey process, you’ll often need to segment your answers into several different audiences.
That’s why it’s so important to enable UTM tags, which can help you gather much of the information you’ll need without having to ask the user unnecessary questions.
By following these basic rules, you’ll be able to create high-quality quantitative surveys that validate your hypotheses and provide representative responses that will be key to improving your product or service.
