This article focuses on how to create and structure questionnaires . To read more about the advantages of conducting user interviews, read my previous blog The Power of User Interviews.
The design and development 0f a questionnaire is the most neglected part of the process of user research, and has been taken for granted by even the best of designers. The rhetoric almost always is that once you know what you want to be answered, the questionnaire would take away the organic nature of the conversation and turn it into a mechanical ordeal. Though I understand this to be a valid concern, it is important to learn how to use a questionnaire to guide an interview rather than using it as a script.
Why Is A Questionnaire Needed?
Before we set out to define how to design a questionnaire, let me first make a case for why it is needed in the first place. The following are some of the more compelling reasons to have a questionnaire before conducting interviews
Makes You Understand The Problem Better
Once the pen touches the paper, the assumptions start to disappear and the clouded understanding gives way to a clearer problem statement and the goals for the research.
Ensures Everyone Is On The Same Page
Having a well-defined questionnaire leaves little ambiguity between team members and results in better communication with the stakeholders.
Allows For Better Collaboration
Since the goals are clear and everyone has the same understanding of the problem, it becomes easier for the team members to share ideas and contribute to the design exercise.
Ensures Everything Has Been Thought Through
A great advantage of having a structured questionnaire is that it helps in covering all the bases. It becomes easier for the team to understand the scope of the interview and add questions/topics that would help them in taking decisions later.
Leads To Crisper Questions
The questionnaire once framed, can not only be worked upon further to translate the questions to their simplest form. But, also the fact that the questions have been framed and do not exist as notions in the heads of one/a few members of the team leads to concise questions being asked during the interviews.
Helps The Conversation Be On Track
While conducting interviews, one often finds themselves in a spot where the conversation is leading itself off track; in such situations, having direct questions for the user helps steer the dialogue back into the intended direction. The questionnaire also works as a reckoner to ensure that everything that needed to be gained from the interview has been gained.
Designing A Questionnaire
Just calling it ‘designing a questionnaire’ would make some people squirm in their chairs; these would be the ones who think of design as something that is visual, and therefore feel very uncomfortable by the mention of terminology such as ‘Design Thinking’. Although since these aren’t the people reading this, let’s go ahead and design ourselves a questionnaire.
Have A Clear Problem Statement
A problem statement is a pre-requisite for designing a questionnaire. A problem is what must have led to a need for research and the choice of interviews, so it is imperative that you have a solid problem statement. If the problem currently exists as mere ideas in the heads of the team, I would recommend putting it down in words and having it framed and put on the walls in front of every team member(maybe this would be overkill… but have a problem statement).
Example: We recently undertook the redesign of the seller panel dashboard of a leading e-commerce platform in India. We started off with defining what we were trying to solve for
“Simplifying the dashboard to it’s core components”
Make A Note Of Your Assumptions
Whatever be the problem, you have a few answers right from the onset owing to your own experiences and knowledge of the field. Take the time to put these down in a document along with the reasons for why you believe them to be true.
The assumptions and prior knowledge were-
- users prefer using dashboard to process orders
- users don’t find much value in financial information
- people do not scroll to discover what’s below the fold
Challenge Your Assumptions
Question the reasons behind your assumptions yourself, and allow other members of the team to review them. This will help you have different perspectives without leaving the comforts of your four walls. This will also enable you to get views of other people on what information would help build a better product. Start thinking in the direction of what you would need to know to solve the problem at hand; compare this to the knowledge that you already have and find out what is missing.
Trying to understand the reason behind our assumptions we asked ourselves-
- why are there 5% clicks on the financial information?
- what makes users process orders from the dashboard instead of the orders page?
- is there a discoverability issue with the information below the fold or do the users not find value in the information?
- what information can help the users grow their business?
Create Knowledge Clusters
Group the information that you need into clusters based on the type of information, and start assigning them importance based on the value that they provide in solving the problem. This would help you concentrate on the most important information without ignoring the others.
We created the following knowledge clusters-
- Order Processing
- Business Direction
- Financial Information
Write Questions For The Perfect World
Having completed all the ground work for creating good questionnaires, you can start with writing the questions that would get you the perfect answers if semantics did not exist and everyone knew exactly how design works. These questions are not for the user, they are for you to base all your questions on. This is what you would measure the success of your interviews against.
The 5 core questions we wanted to be answered were-
- what information do you want on the dashboard and why
- how do you evaluate the health of your business
- what insights would help you grow your business
- how do you use financial information
The questionnaire should start with generic questions on topics that the user is most comfortable with; these are generally the questions which you already knew the answers to. These would help place the user at ease and enable establishing a rapport with the user.
The next line of questions has a lot more room for knowledge expansion. These questions help to dig into the core of the problems the user faces, and what they feel would help solve them. The questions here need to have a series of follow-up questions to enable an objective view of what is plaguing the users.
The last leg of questions involve tangential questions, which appear to have little value in answering the problem at hand. Having said that, sometimes these would end up revealing the most useful insights because your assumptions are revealed to be false here.
To check the detailed questionnaire click here
Conduct Test/Mock Interview
Once you have a comprehensive questionnaire, you need to check if you’ve covered all the bases, and if the flow of the conversation remains organic. The ideal way to test your questionnaire would be to conduct an interview with a real user, but depending on the stakes at hand and the complexity of conducting one, it might not be possible. In such cases, you can conduct a mock interview with someone who is not directly involved in the project.
During the test interview it was revealed that the users were interested in financial summary, but the current presentation was different from what they were used to, and therefore confusing
Based on the results of the first interview, make improvements to the existing questionnaire. One important thing to remember here is that the goal is to solve the problem at hand and not to have the perfect questionnaire, so getting stuck in multiple iterations of reviewing and refining the questionnaire is not advisable.
We added an entire section of questions on what should be part of the financial summary and how can it be presented in an easy to understand manner
Having a well designed questionnaire will ensure that you have clarity of purpose and are well prepared for the research. This would lead to a more productive conversation and the users won’t feel that their time is being wasted.