The Craft of Designing Surveys

Surveys are a cheap and simple way to get feedback from your customers or users. A well-designed survey that delivers a good user experience can ensure not only higher response rates but also data that is both honest and accurate.

Many of you, at least once in your life, must have taken a survey.

Now try to remember, how many of them did you leave midway because they left you mentally exhausted?

hmm… I guess the answer would be many.

And how many were fun and delightful to answer and left you wowed?

Very few, right?

If you think about what differentiated a good survey from a bad one, the answer would be simple.

The good ones were simple and short. The bad ones were just too long and confusing.

Surveys are one of the most used and “abused” modes of collecting feedback from a large audience. I say abused because now-a-days every other app or service wants to know what you think. But the pain they put you through to get that information is agonising to say the least.

You sign up for a new product…. they mail you a survey.

You go to an event…somebody comes up with an iPad asking you to part take a survey.

You order food through an app….and you get a survey.

The biggest mistake that we do when creating a survey is that we forget that it is also a form of conversation. The only difference is that here, the participant has to converse with themselves, introspect and then respond. Your understanding of this mental model when designing a survey will determine its destiny.

Stop creating surveys and start designing them

“Creating a survey is simple. List down all the questions you want to ask and voila!! you have a survey. Well sorry to hurt your feelings darling but what you just did was create a questionnaire.”

A questionnaire by definition is a set of printed or written questions with a choice of answers, devised for the purposes of a survey or statistical study. On the other hand a survey is defined as the measure of opinions or experiences of a group of people.

The difference between creating and designing a survey is that when you design a survey you think beyond just the questions you need answers for. When designing questions you think about the channel and context in which the participants will take the survey. Answer types are decided based on how you want to interpret and represent it to your stakeholders. Holistically speaking when designing surveys you design a survey experience that influences participants to share their thoughts and opinions in a frictionless manner.

So how can you design an effective survey experience? Here are some key pointers for designing a survey.

Do your homework

Creating surveys that require users to write essays for answers should be avoided as much as possible. Research shows that surveys with more objective questions have higher response and completion rates as compared to those with subjective questions.

This brings up another problem — How can you provide participants with relevant answer choices?

The best way to get it is to invest some time in doing a bit of qualitative research. After you have identified your target audience and finalised the questions, reach out to 5–6 people and engage them in qualitative interviews. Answers that you get from this set will become repetitive after you have talked to the 6th person. Apart from slight variations, you should be able to see trends in the answers. Now use these variations as answer choices to understand the pulse of the larger group.

Stop interrogating and start speaking

The more natural you make this conversation, better are your chances of getting honest and accurate responses. Creating questions in this way helps you think of answers that are more relatable to the participant thereby helping them answer faster and giving you more accurate answers.

For instance, let’s take the example of asking someone about how often they change their display pic. You can do it in two ways.

You can ask them how they do it. Straight and simple. Yet you can screw it all up by giving a set of non-sensical options for answers.

Or ask it in a manner that matches with the way someone will think when asked such a question in a conversation.

Asking questions in first person provides participants an experience that will help them re-imagine the situation better and answer accurately.

Design answers that can be analysed

Surveys provide quantitative outputs which when represented right can paint the right story about your customers. But to be able to do this, survey answers need to be designed to deliver meaningful and actionable insights.

Now let’s take the case where you want to understand how likely is it that a prospective user might buy your product.

For starters, a user will never think in the way the options are provided. Secondly what would be my inference if I have 30% users saying “Not so likely” and 12% saying “Not at all likely”? Is it that there is a slim chance of selling my product to the former group?

After I first posted this blog I had a lot of people asking me “How will a question like this entice an honest or rather a reliable answer from the participant? There is no need for them to remain consistent to their answer.” I agree, it is indeed unreliable but the solution for this is to not ask the user about their opinion and ask them about 2 things.

• Their behaviour in the past (in a similar context)
• Things that might influence their behaviour (reviews, recommendations etc)

Here if a user has no intention of buying it he/she can very clearly choose the relevant answer and move forward to next question. The benefit of this approach is that you talk the user’s language and help them share their answer with the least amount of friction. And when analysing the responses there is no ambiguity for you to come to a conclusion about your audiences’ intent.

Keep it simple, keep it short - Respect your audiences’ time

“If your survey is lengthy your audience is just filling it not answering it”

Understand that your users are not obliged to give you honest answers. If you put them through a gruelling session of 30 questions in a survey, you are immensely reducing the chances of people completing the survey. And even in the completed ones the chances of people having chosen random answers will be very high. Respect the fact that your audiences have other things to do in their lives apart from answering your survey.

Ideally surveys should not have more than 10 questions. Time and again I have seen completion rates that are way higher for surveys with less than 10 questions than otherwise.

Do your research well on the audience from whom you seek information. For instance, if they are your customers, use your CRM data to identify their name, company and other such demographic data. Stop asking questions for which you already have answers. It will only lengthen your survey and reduce your chances of getting data on questions for which you actually need answers.

Design for the context

Keeping in mind the context in which your audience will be taking your survey can make your completion rates go through the roof. Imagine you have a sales CRM product and you want to get their feedback on the product. Giving a popup quiz with 10 questions at their peak work time will only hinder their usual work cycle. At the same time sending it to them, at say post their lunch time, will get them to take and complete the survey without disturbing their day-to-day work cycle.

Put yourself in their shoes and imagine what should be running in their when they take the survey. Then identify when will your audience be in this state of mind on any given day. Time your surveys to reach them accordingly.

Conclusion

A deeper understanding your audience will help you craft the right survey experience for them. And well crafted survey that matches the experience will give you insights that will be much deeper and actionable.

Stop creating surveys and start designing them.

Originally published in Freshdesk Blog.