Q&A: I’m doing a “UX survey” for my team, am I asking the right things?
On how to formulate a survey.
From time to time, I get asked about how to do certain things. And, I figured it would be helpful to more people to share these steps or tips with others. This Q&A series is based on those conversations.
Interestingly enough, in the past months, a beautiful thing has been happening — more people have been asking about how research is conducted!
Why do I love this? Because I love seeing people make the leap from “theoretical” appreciation, to hands-on application. Yay!
More people asking about the process means more people having the balls to learn and do it for themselves!
I stand by what I said before — what people don’t immediately realize is that the “user” in User Experience means “getting to know your users”, i.e. user research and analysis.
So, for everyone who’s asked me for tips about questionnaire construction, props to you.
A friend who works in the digital strategy field asked for tips on formulating an effective survey.
She already had survey questions in mind, but just needed to check: if there was a way to ensure it was a good set of questions.
We worked on improving her survey flow by figuring out the following questions:
Let’s start with the context. (Of course, details have been omitted to protect the specific case study.)
Q:
The website isn’t developed yet. But, its purpose is pretty straightforward:
1) Omni-presence and branding (Brand has no corporate site established)
2) Digital library (It’s a…brand with loads of recipes offline…that they can utilize online.)
3) Extend the reach of their cooking show (…house the content, in case someone searches for them…)
Given that those are the main objectives and the content we have, I kind of have to work backwards and get to know the consumer again. I guess to validate — the process is a bit reversed because we had the website features in our head, before talking to consumers, because we based them on best practices.
Anyway, I just need help on how to formulate an effective survey given that there isn’t a website yet, so it’s not something that we can ask users to evaluate.
Are there template surveys for website planning?”
A:
1. What will you use the survey results for?
For example: is it an internal report for the Creatives — or something to present to Client?
Q: We just want to make sure that what is in the website fits what the target market is looking for and how they use the site.
Note #1: Check the purpose (action points) of the research results.This is the first question to ask yourself, when refining a questionnaire because the needs of an internal team (for research), differ from the needs for a client presentation (e.g. let’s face the harsh truth, agency client pitch research sounds more like idea validation than anything else).Once you’ve worked out who the research presentation is for:
A: Okay, as the strategist, …
2. What elements do you need to know to be sure and confident about what it is that you’re trying to find out?
Note #2: Consider all the elements (information) you need to feel confident about deciding on the research’s purpose.This directly impacts the kind of topics you’ll look for.
Q: “I need to know what motivates them to look for recipes online –Like, why would they opt to look online versus just ask their friends?”
A: Perfect. Then you can craft your survey around that. At least, that helps you zoom in on the questions you need — to understand that [motivation].
Q: “Can I get specific right away — like ask about online search behavior straight from the get-go? Like, if it’s better to ask open-ended or multiple choice?
Note #3: How are you administering the questionnaire?Because that has an impact on question flow as well. The key is to keep it as simple as possible - especially if it’s an online survey, that can they leave at any time, with no (carrot). Simple, meaning the less long open-ended questions, the better - like multiple choice, or fill-in-the-blanks.
A: It’s a web survey, right? Like a questionnaire.
You may need to keep it simple — fill-in-the-blanks or multiple choice
Then, you can…
3. Progress to your most important question
(“Why did you go online instead of other sources?”)…
…By going general first (where do they learn to cook food?) and then becoming more and more specific (Why did you go online instead of other sources?)
Note #4: Pinpoint -- What is the most important question you want to ask?Q: I was going to ask about websites right off the bat.
A: Technically, you could. In a non-survey set-up. But, surveys typically flow from “generic” to specific, because participants will be biased if you ask direct questions from the get-go.
Q: For SEC (Socio-Economic Classification), do I just ask them their income?
Bonus: Can you ask about SEC on an online survey?A: Legitimate SEC calculation is tricky (it’s a mix of a lot of things — house quality, education, income of the household head, etc.).
What kind of SEC question would you like to know …
as a strategist? Since that’s more for your team benefit — what level of question would you need to determine that that participant is part of your target market?
Q: Well, I don’t really need to dissect their SEC. …just to know if they’re part of the target market.
A: I personally recommend “lifestyle check” questions. Meaning what types of activities, purchases would let you know a person was “A, B or C”? But, if you want a quantitative question, then, yes, income IS a safe bet.
Is this an anonymous survey?
Q: Anonymous :)
A: Oh, then, yes, people might put in their income, so you can try.
Q: Oh wait, just remembered, I’m sending it out to housewives. So income might not be a good question after all.
A: Well, yes, that could be trickier. Unless you ask about “household income” — which is everyone’s salary (in the household) put together.
Q: Right, okay, I’ll do that.
A: One clarification — you might need to ask how many they are in the household, then. Because someone’s household income may look low, but it could be because there are only two of them living together in the house.
If you’re writing your own survey, and you don’t know if you have the “right questions”, then, hope this Q&A helped.
Side note, I’m a bigger advocate of doing usability testing, or looking at analytics data than quantitative “self-reports” (e.g. surveys) — when designing digital products. This is because we sometimes can’t articulate how we behave online (“What you say isn’t always the same as what you do.”) Just something to consider :)
Originally published at www.angelaobias.com on May 13, 2015.
⏩ 👫 If you found this helpful, please share it to the people it can help, too.
As I mention in weeknotes I write, I’m a fan of designing in the open.
If you want to follow the things I’m learning along the way, as a researcher, strategist and teacher, follow me on Medium or subscribe to our yet-to-be-regular newsletter.
