What is your preferred survey bias?

Think back to the last time you wrote a survey. What percentage of the questions were ‘required’ and what percentage were ‘optional?’ I’m going to take a wild stab in the dark and guess that 100% of the questions were required.

Why?

Why is it absolutely essential that every question be answered? I’ll tell you why. Because data analysts don’t like working with missing data, and because we like to say that we had 300 completed surveys not 297 completed surveys. A number that ends in two zeros is clearly a superior number. Although, of course, a number that does NOT end in two zeros suggests superior precision and attention to detail. So, six of one…

Do you know the result of forcing a response to every question?

Bad data. If you’re going to force me to answer a question I don’t want to, or can’t, answer, chances are the answer I end up giving will be worse than if I hadn’t answered it at all. And, how exactly are you biasing your results if you only accept data from people whose personality types mean that are willing to suffer through that monstrosity of a survey you’ve cobbled together?

I will admit that I’m mostly guilty of this. I, too, love to see an answer to every single question in my survey. I hate dealing with missing data. Why is it missing? Did I miss an important answer? Was my question unclear? Did the question not apply to the person? Was my question offensive or insensitive? Was the question too complicated and difficult to generate an answer to? Maybe the person was just being lazy. Maybe it’s not really my fault at all! (Dreamer!)

In most cases, the problem is me, us, the survey writers. If we wrote simple, clear, uncomplicated questions that included every possible answer, then people would be able to provide a good answer to every question. Missing data wouldn’t be a problem. Our statistical techniques wouldn’t have to accommodate missing answers with inferred data and smaller sample sizes. Sometimes, as much as we hate giving people an easy out, it simply means ensuring we’ve included every possible answer such as:

• None of the above
• Other
• Does not apply
• 7 or more
• 2 or less
• 0

Until we get there, let’s take a second look at the survey we’re working on right now and simply consider whether every question MUST be answered.

Originally published on Peanut Labs. Email Jonathan to set up your full-service or DIY sampling portal and instantly access millions of double-opt-in, pre-screened panelists, or Davis to find out how you can monetize your loyalty, e-commerce, or gaming website.