Best Practices in User Survey Design

One of the most invaluable processes to guide product development is that of the user survey. Surveys are ubiquitous: you fill them out when going to the doctor, when checking-in at an airport, when rating a pair of shoes you bought online. But they often may feel tedious, uninspiring or right-down useless. If you aim to collect actionable data from your target audience, you will need to master the details of user survey design.

Know What You Want to Know

All surveys need to have a well-defined goal, a burning question that you need to answer, a hypothesis you want to validate. Usually, these topics come from formal or informal user interviews and qualitative processes. In the survey, you want to use mostly quantitative methodologies to collect and stratify data that will help guide your decision making. So, before you start sending out that questionnaires, do a small survey on yourself, by answering the following questions:

• What do I want to know?
• Which answers do I need to know it?
• Who should I be asking the questions?

If these points are clear in your mind, then you are in the good path! Let’s dive a bit deeper on the last topic.

It is quite possible not everyone will be able to answer your survey questions in a clear, unbiased way. When setting up your survey, do not forget to spend time thinking about who your ideal audience should be. For example, employee happiness surveys are very popular, but there are several biases that come with quantifying a very subjective idea. For starters, more recent employees may have a tendency to be in a honeymoon phase and give more positive reviews, so you should consider adding questions to your survey that would help you categorise respondents by tenure time — or avoid adding very new employees altogether.

Tools of the Trade

In the end of your survey, you should be able to identify groups of respondents that have a certain behaviour or can guide your decision making process in a certain way. For this, you can employ a various set of questions for different situations:

1. The binary question: very useful when guiding respondents through question sets and to do high-level stratification of your audience.
2. Multiple choice: used to collect guided feedback. When you use multiple choice in your survey, be sure to either cover all the possible answer space or, when impossible to do so, cover the answers you deem most likely, while adding an option to answer with ‘other’. Adding a small free text option here is also a nice thing to do.
3. Rating scales: an all-time favourite to gauge respondents’ perception about a certain product or activity. With a scale, you try to quantify a feeling towards a certain action, and it is very common to encounter rating scales in product surveys like the Net Promoter Score (NPS).
4. Likert scales: similar to numerical rating scales, these scales instead use sentiment groups as answers, usually ranging from “very dissatisfied” to “very satisfied” with 3 to 5 in-between options. These are good scales to use on product feedback, for example.
5. Demographics: you will usually need to stratify your audience, so try to place in a few demographics questions around appropriate segments. Be aware that, for data privacy reasons, you should never drill as deep as to be able to individually identify a respondent. However, depending on the purpose, demographics questions with age groups, employment situation or gender are commonly used.
6. Open ended questions: do you feel like giving your respondents a chance to provide additional unstructured feedback? Sometimes analysing open-ended answers can help you formulate yet new hypotheses to test, so be sure to include a final feedback question on your survey whenever you have the need.

Engage your Audience

Many people will avoid answering your survey if it feels too long or is riddled with ambiguous questions. A rule of thumb is to always make the survey as short as possible, for what you need answers for. Remember that your time is valuable, but so is everyone else’s!

Additionally, you can rely on a few engagement techniques to ensure you get as many quality answers as possible:

1. Be clear about the purpose of your survey: people like when you are upfront and will generally be happy to take one or two minutes of their time if it will help improve a service they are already using.
2. Throw in an incentive: if the survey may feel particularly daunting or complex, you can add some other forms of incentive such as small discount vouchers. It is certainly not an appeal to altruism, but can work very well, especially in on-site surveys and usability tests.
3. Be clear in your language: don’t use overly long expressions and keep the language simple. Questions should be easy to understand and do not demand a lot of cognitive effort from your audience.

Dos and Don’ts

So, you are ready to throw your survey into the wild! Well done. Let’s just do a couple of checks before that, shall we?

One of the major driving points for a survey that converts and yields results is to treat it like you would a conversation: be open about what you want to achieve and don’t get personal too early. If you need to stratify your data, especially with sensitive questions about gender or sexuality, try leaving those to the end of the survey and provide an “opt out” answer. A very common mistake survey writers do connects with the natural tendency to lead your audience towards the answers you expect to get. For example, do not use leading language such as “What do you think about the seating in our amazing new card models?” — you are already implying the cars are great, and thus so will be the seats. Whenever possible, use neutral language and let the respondents do the judgement.

As mentioned above, scales are a great way to quantify a person’s sentiment about a product. However, you should always ensure that, when you use scales, to provide an equal number of positive and negative choices, with a neutral option in between, otherwise you may be biasing your answer towards one side of the emotional spectrum.

Finally, keep in mind you will have to analyse your data at some point. Your questions will be useless if the answers you get are not clear and actionable, and there is no bigger offender in this field than double-barreled questions: trying to get one answer for two questions in one go. These questions should always be split into their components, even if it makes the survey just that little bit longer. For example: “Are you satisfied with the quality and price of our pizza?” would work much better with a separate question for price and another for quality.

Go Forth and Gather Data

Taking the pulse of your audience, customer base, product users or office colleagues is essential to develop improvement strategies. Summarising, your surveys should:

• Have a clear and transparent goal
• Be directed towards a representative, unbiased audience
• Use simple language and ask unambiguous questions
• Be as short as possible and straight to the point
• Provide the right level of audience stratification so you can identify group patterns

Have fun in your discovery journeys!

Interested in joining our team? Browse our open positions or check out what we do at HeyJobs.