Moderating Best Practices

6 min readAug 9, 2017

A guide for beginners or experts refreshing their interviewing chops

I’m obsessed with moderating, and I think anyone conducting qualitative research should be too.

Moderating is a systematic and structured retrieval of information. It is not a conversation. Moderating is used to gather information about behavior, motivations and beliefs from participants and display that data to stakeholders (into the observation room and your findings). It is part investigative, part display and always strategic.

The guidelines and principles below are geared towards user studies, but can also be applied to any other type of moderation (e.g. in-homes, etc).

______________________________________________________________

Interviewing logistics

Be prepared to multitask x1000: Moderators must balance multiple internal questions at once, including:

How much time is left?
How do I bring up the topic we passed?
Is the participant comfortable?
This seems similar to another participant. How can I make sure? Is the camera picking this up?
Was that clear for the observers?

To navigate these internal questions be prepared. Have easy access to a clock, keep your interview guide handy, and keep notes of questions to go back to or dig deeper.

Be broad before specific: Make sure your interview is structured so that the broadest questions are asked first, and the meatiest questions are asked last. Once you introduce a specific topic, it may be impossible to ask more general questions without the responses being colored.

______________________________________________________________

Approach

You are the apprentice; the participant is the expert: During each session, consider your participant as an expert, there to help you learn. She is an expert on behaving and interacting like herself, which is exactly why she is there. Your job is to learn from her (not teach, change her behavior or persuade her).

Make the participant comfortable: You are the participant’s advocate. Your job is to help the participant clearly articulate their motivations and beliefs. Therefore, it is essential that they know:

You did not create, design or inform the product they are experiencing (even if you did).
They are not in a test, and there are no right or wrong answers. They cannot hurt your feelings
Give the participant agnostic reinforcement, validating their participation, not their “answers.”

Bad example: “You’re right. That is really hard.” “You’re really good at figuring this out- other participants have had a lot of trouble.”
Good example: “This is great feedback!” “Thanks for sharing, that is really helpful.”

Leave your assumptions at the door: You may have hypotheses, but your role as a moderator is to do everything possible to make it clear whether the participant validates or disconfirms those hypotheses.

Don’t assume anything, which translates into- don’t be afraid to ask even the most basic of questions. Basic questions that requires participant articulation may include asking the participant to:

Articulate why they are doing the behavior they are doing
Elaborate on a belief or motivation they express
Answer the same question they have answered previously, but this time in a different way. This is to ensure you are getting a reliable answer

Pretend you’re in the observer room: Be conscious of how the interview is being perceived by your stakeholders. A good rule of thumb is to put yourself in the shoe’s of your stakeholders in the observer room (or watching on camera) and ask yourself the following questions:

Did the participant clearly confirm or disconfirm the hypotheses? Does the most skeptical stakeholder see what you see? Ask the participant again so that it is abundantly clear to the observers
Is the participant speaking clearly or loudly enough for the video to pick up? If not ask the participant to repeat himself or speak up

Stay out of the research: Remember that while you are moderating your job is not to provide customer support. Though it’s tempting don’t: Answer customer support questions during the interview or help a participant through a task. Instead defer any customer support help to the end of the interview. Make sure it is exceedingly evident that the participant cannot complete a task: Ask the participant several different ways to complete the task until conceding that they cannot; only then give them a hint.

Good example: “That’s such a good question and I’m so glad you asked it. I’m going to leave time at the end of the interview for us to talk about this more.”

____________________________________________________________

Elicitation techniques

Answer questions with questions (not answers): In almost every interview, the participant will ask you a question (e.g. “Is this how it works?” “Did I do that right?” “How are you suppose to do that?”). Though it’s a natural reaction to reply to a question with an answer, don’t. Instead, reply to a question with another question

Good example: “What do you think?” “How do you think it works?”

Always use open ended, non-leading questions: Qualitative research is your chance to hear directly from the participant in her own words. Therefore it is crucial to make sure you are doing just that, and not limiting the participant’s responses with yes or no questions, or closed ended questions.

Bad example: “Do you think shopping this way was hard or easy?”
Good example: “Tell me what it was like to shop this way.”

Easier said than done, but stay far away from leading questions. A leading question not only can bias your participant’s response, but also is easily detected by any observers and may influence the credibility of your findings.

Bad example: “You looked like you had trouble using that. Did you?”
Good example: “What was it like to do that?”

A leading question can even include a single word. For example, try not to direct the participant with language that is represented in the prototype

Bad example: “Let’s say you wanted see your bookmarked photos. How would you do that?” (when the button is labeled “Bookmarks”)
Good example: “Let’s say you wanted to see the photos you saved. How would you do that?”

Mimic the participant’s language: Echo the participant’s words, rather than bringing in your own. This may mean saying words incorrectly, or even words that don’t exist. This will make the participant feel more comfortable, but perhaps more importantly, provide product recommendations. For example if the participant calls Pinterest “Pin Interest” or a jet-ski “jetter-ski,” then you should too (both true examples).

To make sure you are capturing natural language occurrences after the interview, it is very important to take verbatim (or as close as possible) notes during the interview.

Use silence: Silence is a powerful conversation tool. When faced with silence, almost everyone feels uncomfortable and quickly wants to fill it. That’s why it works as a tool (and why it is hard to do!). This strategy is particularly useful when a participant is either closed up or needs time to think.

There are some “silent prompts” you can use too.

Good examples: “Because…” “When you…” “Then…”

Ask the participant to recap rather than doing it yourself: Sometimes you may want to recap a bunch of things the participant said, and make sure you and your observers are understanding correctly. When this is your goal, instead of being the one to recap, ask the participant to do it for you. Otherwise, this will fall into the category of the dreaded leading question.

Bad example: “It sounds like what you’re saying is x,y,z.” “What I’m hearing is x,y,z.”
Good example: “That is so interesting. Can you try to summarize that for me?”

Be yourself: Though the moderating room may bring out a different side of you, and you will need to use conversational tactics that you might never think to use in real life, don’t forget to be and leverage yourself. We all have different styles and it is important to make sure you are comfortable moderating. Each of us has our own traits that we can leverage in the moderating room.