How to Conduct Proper User Research Design for Conversation

User Research Design is the foundation of understanding how to structure conversation design through research, planning, and testing. What is the number one skill you must have in order to create great software? It’s not a great memory, a mathematical mind, or even the ability to code productively for 48 hours on little sleep.

What you need most is empathy. The rest comes with experience.

Empathy is the first foundational step in Design Thinking, an approach to problem-solving that has been behind commercially successful technologies from Apple to Uber. Design Thinking begins with the end user and every step along the way is centered in responding to human needs based on user feedback. After all, what’s the point of building something that nobody wants? That may sound simple, but the incredibly high startup failure rate is a testament to founders who preference product over market.

At Stanford’s Design School (d.school), Design Thinking was formalized into a set of principles and guidelines to help structure the process of conducting user research, writing out the problem definition and then opening up the creative floodgates of designers to make something that is both radically new AND directly relevant to the end users.

Here are a handful of observations on what makes user research for conversational design projects wholly unique.

1. Investigate how your specific users talk to your brand (or a brand similar to yours)

Define your customer persona as narrowly as you can. It may seem counterintuitive, but the more you narrow down your customer persona, the wider your appeal. Something that is meant to appeal to everyone ends up pleasing no one. Find a close competitor or a company in another industry that serves a similar customer segment and interact with their chatbots.

A potential user survey, using low-cost services like SurveyMonkey or Mechanical Turk, can be an effective starting point. Responses imply how they expect their customers to respond.

The next step is to search through any material you have such as customer service logs, frequently asked questions, online user review sites or comments sections on niche blogs. Make note of common themes, unspoken assumptions, and end goals.

2. Define your brand’s value from the user’s point of view and deduce their primary motivation

After accruing sufficient material to truly empathize with the user, your second step in Design Thinking is to tightly define a problem statement as a user story. For example, “I want to build a customized drone that will do what I tell it to do.” This problem statement covers:

  • Who is the user? A beginner personal drone pilot
  • What is their motivation? To enjoy mastery with less difficult learning curve.
  • What do they need? Special customized controls that make it easier to pilot the drone.

User stories can include real quotes, persistent beliefs, and emotions. In the example above, the users may be frustrated or frightened by complexity when they interact with the chatbot. The user’s typical emotional state will inform the central pathways of your conversation tree.

3. Start the ideation process with a specific user persona in mind

Your user persona should include representative demographic info, a handful of their top goals and what they want to take away from the customer experience. Write out pain points and group them by categories, such as “logging onto the website” and “Understanding the controls.” You can rate pain points by severity and complexity in solving them. This is the ideation phase of the Design Thinking plan, and it crosses over into the Prototyping phase. It’s important to realize that this is not a one-directional pathway. If you don’t feel like you’ve gathered enough information, go back to the problem definition stage and gather more data by alternate means like phone interviews. If the prototyping exercise does not produce the desired effect with user groups, cycle all the way back to the empathize stage to broaden your understanding of what users really want.

user research design

4. Complete the Build-Measure-Learn loop

One of the areas of overlap between Design Thinking and the Lean Startup methodology is the final stage of iterative improvement. In the popular press, this is sometimes referred to as “Fail fast and fail cheap.” Moving from the prototyping stage to the testing stage usually involves collaborating with users on new ideas, which moves you back to the ideate stage. Once again, you select only the most promising idea to prototype and move quickly into a testing environment where you can get feedback inexpensively before you roll it out the wider public.

User Research Design strategy checklist

  • Customer Persona — be specific
  • Comparing other companies’ customers and tactics (benchmarking)
  • Gather user primary data — conduct surveys through Surveymonkey or Mechanical Turk
  • Look through previous feedback and data
  • Define a problem statement as a user story
  • Prototype with a user persona in mind
  • Complete the Build-Measure-Learn loop

Best practices in conversational design

This is the right time to review the 5 Rules of Chatbot Design, loosely based on Shneiderman’s 8 Rules of UI Design:

1) Be consistent — User trust in the chatbot relies on it behaving and speaking similarly in similar situations. Use consistent color schemes and consistent vernacular. Don’t lose sight of this as versions upgrade and various teams work on different limbs of the conversation tree.

2) Build in shortcuts for frequent users — Reward those who use the service often with suggestions on how they can move more quickly and easily to their destination. Function keys, emoticons, macros, and customized abbreviations make a cleaner interface.

3) Design dialogs to follow a 3-stage plan — Organize dialogs and likely user pathways to have a beginning, middle, and end section to give the users a sense of completion. It allows them to clear their minds and offers a sense of relief.

4) Build ‘undo’s and confirmation notices at critical points — Users should feel confident that anything can be undone, including adding extra layers of reversibility at decision points that generate anxiety for users.

5) Put control in the hands of the user — Frame dialogs so that the user is the initiator of actions and the chatbot is there to provide assistance. Phrase commands in the user’s point of view and be as brief as possible.

Remember that these are only best practices and you are better off breaking any of these rules than designing something that is not engaging for your users.


Originally published at Botsociety Blog.