Are you sabotaging your own user experience survey?
How to craft better, more effective survey invitation emails.
Surveys are wonderful tools for collecting quantitative and qualitative feedback from your users. You’ll get hard numbers to share with your team to back product and design decisions. And you’ll gain insights from open-ended responses that you can act on right away.
So you did the legwork. You designed and wrote a sound survey that will provide you with valid, useful data. But your survey doesn’t begin with your survey. It begins with the invitation you send to your users asking them to fill out your survey. And a poorly-crafted email invitation can sabotage a great survey.
Here are five tricky techniques you can use to sabotage your own user survey with your invitation email! (Along with some ways to avoid it, and a template you can use to craft better emails).
🔪 Sabotage Tactic #1: Skew your results before the survey begins
Messaging or imagery in your email invitation can introduce bias into your data that skews results before the survey even begins.
This invitation, from a condo developer, is curious whether future buyers would be more interested in a pool or additional parking in future developments.
Unfortunately, the email introduces bias into the results right from the get-go.
The subject line “How much would you love a pool?” can introduce voluntary selection bias into the study. Readers who love pools are more likely to open the email and complete the survey than those who would appreciate more parking. This bias will be reflected in the survey results.
The email also introduces bias by turning the invitation into a leading question. As humans, we’re influenced by emotionally-evocative imagery. This email has two glamour shots of pools, and a single, boring photo of a parking space. The options are described as “a resident-only outdoor swimming pool to enjoy,” or “an additional parking space.” The email invitation itself leads participants towards a single “correct” answer.
This email may prove to be an expensive mistake if they build a pool when condo-buyers would actually prefer additional parking.
How to fix it
- Make sure you’re not accidentally creating leading questions through your email’s messaging . Be objective and use objective language.
- Avoid putting questions from your survey in the subject line.
- Avoid using imagery tied to your survey’s questions. Company or product logos are okay.
💥 Sabotage Tactic #2: Slash response rates and introduce bias with writing that readers don’t read
Users are busy humans who have things to do. There’s no better way to slash your response rates than to write a clunky, formal invitation email.
The average American reads at a 7th or 8th grade level. This email invitation from Seagate scores in at grade 12.8.
The result? Less readers will read your email and click through to your survey. And those that do are subject to selection bias. There’s a limited subset of readers that will push through.
How to fix it
- Use human language in your invitation. Keep it simple. Check it against reading level calculators.
- Manage the four dimensions of tone in your messaging: funny v.s. serious, formal v.s. casual, respectful v.s. irreverent, and enthusiastic v.s. matter-of-fact.
💣 Sabotage Tactic #3: Overwhelm your users with information or instructions
Your invitation email has a single job to do: get your users to click through and begin taking the survey. Overload your email with info or instructions up-front and your readers will move on.
This email asks readers to read a wall of text and make a deliberate decision up-front about whether they want enter a draw. Then, it provides specific instructions to remember and follow once they complete the survey.
How to fix it
- Keep it simple.
- Provide only the information that readers need to know to get going. Context is useful, need-to-know information, but know when to stop.
- The only instructions should be how to access the survey, and how to contact you with questions. Your survey can provide the rest.
🔫 Sabotage Tactic #4: Show users how little you value their time
Studies show that personal incentives can boost response rates. But those same incentives can imply you don’t value your user’s time. This could result in lower response rates than if you’d offered nothing at all.
This email asks readers to dedicate 15–20 minutes of their valuable time in return for a small chance of winning a $50 gift card. If the incentive’s value is out of line with the effort asked of the reader, why would they take part?
The amount that you offer to users as an incentive depends on:
- Your budget
- How you provide the incentive
- Your target audience
A good incentive balances these three factors. A neurosurgeon would pass on a chance to win a $25 gift card, but it may entice a student. If you really want to hear from a neurosurgeon, plan to fork over $200 in exchange for their response.
Science says that people respond to surveys for one of several reasons:
- They want to be helpful
- They enjoy the topic of the survey
- A tangible benefit
With this in mind…
How to fix it
- Offer an appropriate incentive for your specific audience. This may mean increasing the budget.
- Craft your messaging to focus on how valuable their feedback is. Consider omitting the incentive altogether.
- Tie the survey’s results to the user’s experience with your product or service. Let that be your incentive to them.
💀 Sabotage Tactic #5: Make the design “pop”
Your email survey invitation isn’t traditional email marketing. Your goal isn’t to drive sales: it’s to drive participation. Forget to design with purpose, and your engagement will plummet.
This email from a sports governance organization is 90% logo and 5% survey invitation. The type is too small, with a low contrast ratio that makes it harder to read. And it’s stuffed with irrelevant links to other content.
This invitation is hard to read and makes it easy to get distracted. The result: lower response rates.
How to fix it
- Use design to convey tone. Aim for a feeling of lightness and minimalism, which will make the survey feel like less of a “burden” to complete. A text-only email is quite okay.
- Pay attention to detail. It conveys professionalism on the part of the team behind the survey. Why should a user trust you with their feedback if you’re okay with delivering a clunky email?
- Make sure your design is easy to read, accessible, and provide a clear call to action. Your goal is to have them click through to the survey. The link should be the most prominent visual element in the email.
- Avoid opportunities for distraction. Don’t include links to social media or other destinations in your design.
Here’s a simple template for writing effective email survey invitation emails
At Caribou, we’ve created and refined a simple email invitation message that works well for us. Inspired by Quartz’s work getting busy leaders to respond to their survey, we based this template on three principles:
- Provide context. Give people an inside, honest look at what you’re up to.
- Make it personal. Give people a reason to want to get involved.
- Be transparent. Be accurate about how long it will take, what they can expect, and include a means to contact you.
Replace [variables] with your own words.
Have five minutes? Help [organization name] [outcome]
We need your help for just five minutes
[Organization name] is undertaking a [project or activity] to [outcome] for [type of user]. To help us do this, we want to better understand the needs and desires of our [user base or community]. As a member of [user base], we want to hear from you.
If you have a few moments to spare, I invite you to fill out our [survey name or description] survey. This survey should only take five minutes of your time. It will be invaluable in helping us understand how to [outcome]. As a thank you for your time, [incentive offer if available].
Thank you for your time and input. Please feel free to reach out to [the project team or yourself] directly at [email@example.com] with any questions or feedback.
[Your Job Title]