Writing Good Surveys Is Harder Than You Think Pt 2
Part 2 of 4: Structuring the survey
Survey structure
Top and tail
The start of your survey is key to setting respondent expectations and enticing them to take part, while the end of your survey is about making them feel good about it. Both are important.
Provide a short statement up front of what the survey is about, so the respondent knows what to expect, and let them know how long it will take. This will help completion rates, as respondents have realistic expectations of the time investment they have to make.
Don’t forget to thank respondents for their contribution at the end. Some survey tools offer a default thank you message, but a more personalised one will be much better appreciated. You may also consider letting respondents know what is going to be done with the data.
If you’re incentivising the survey, you’ll need to tell respondents how and when they will receive their incentive. This usually requires some terms and conditions around it — consult your legal team.
Your survey is a reflection of your brand. The small touches count. If you don’t bother with them, you’re less likely to get respondents taking part.
Question order
Try and provide a logical flow to your questions to make progress through your survey streamlined and meaningful. This has to be weighed up with consideration of the learning effect and the order effect.
The learning effect
There is a learning effect in everything — the longer you do something, the better you get at it. It’s only natural. This also applies to surveys. Responses to questions at the end of a survey are likely to be influenced by responses to questions at the beginning. So… you have to keep this in mind when you’re interpreting the data. You can, however, use the effect to your advantage e.g., asking the same questions before and after a stimulus to see if the stimulus made a difference.
The order effect
The order effect can come into play if you order your questions a certain way. For example, if you show respondents Image A, then Image B, then ask some questions, you can’t be sure that the image order hasn’t affected the responses.
This is why you need to consider randomising the order in which you sequence certain questions. For this example, you would show half of the respondents Image A first, then B and the other half Image B first, then A.
Some survey tools have a randomisation option, however you may need to check how it works as it may apply to all the questions in the survey.
Avoid reliance on memory
Ideally you won’t make respondents have to remember something from an earlier part of the survey. Repeat text if you have to. Respondents will drop out or make it up if they have to remember too much.
Demographic and other questions
Ideally demographic questions or questions not directly related to your brief are optional, unless you need them for your analysis. Putting a nice note around these, telling respondents that they don’t have to answer but it would be helpful if they did, may increase your response rates.
Placing these sorts of questions at the end of the survey is less threatening and may help response rates too.
Remember to be inclusive; not everyone fits neatly into traditional demographic categories like gender.
Try and avoid questions that will compromise the respondent’s privacy or make them feel uncomfortable about participating. If you do need to know private information, provide assurance that the information will be protected, and will only be used for the purposes of the survey.
Choose your questions carefully
Be judicious about what questions you include. Make sure every question in your survey is there for a reason. It can help to go through each question and ask yourself what you think you’ll learn from it.
Conversely, ensure you have included questions that will address all the aspects you wish to find out. Go back to your objectives — you should be able to map each of them to your survey questions.
