12 user research lessons I learned by mistake
As a UX Designer at iZotope, a tech company that creates audio-centric software and hardware products for music and film, I work on a small design team where designers run user research sessions for their products. I talk to one or two customers per week on video about their use of the Spire Studio multi-track recording hardware and app. Here’s what I’ve learned:
12. Don’t be the designer (even if you are).
In my early sessions, I told users upfront that “I’m on the design team.” Their reactions tended to sound like “Oh, so you make this?” followed by forced compliments and restrained criticism.
To get more candid feedback, my team’s user research guru Mike McNasby suggested a slight adjustment to my introduction: “I’m on the research team.” That simple change has led users to talk more openly about what aspects of the product they enjoy and how their experience can be improved.
11. Know who you’re talking to.
I’ve had conversations cut short because of important phone calls, screaming babies, etc. Users are real people with jobs, families, and precious time. The longer a session, the more fatigued the user and researcher become.
To get the most valuable feedback as quickly as possible, learn about the person you’re going to talk to. If your app or website collects user data, look at their recent behavior. If they’ve taken one of your surveys, skim their responses. I’ve found that 5 minutes of prep cuts out 15 minutes of questions and often uncovers an insight that sparks a conversation.
10. Prepare for the inevitable failure of technology.
Murphy’s Law loves to interrupt the flow of a user session. The video is glitchy. Or the audio is is stutter-y. Or they’re using headphones and I can’t hear them, but when they take out their headphones they can’t hear me. Everything that can go wrong…does go wrong.
In public speaking, they say “fake it ’til you make it.” When video isn’t working, call them on the phone right away, start recording the audio on your computer, and keep the conversation going while you troubleshoot. If you can’t get video working within 5 minutes, cut your losses and stop trying. A meaningful phone conversation beats 30 minutes of failed video attempt.
9. Start at the beginning.
When I began testing clickable prototypes with existing Spire app users, they seemed confused even when the design barely differed from the Spire app they were used to. My team’s user research guru pointed out that the issue might not be bad design, but lack of context.
To get users in the right mindset, create prototypes that include the steps they have to take before using your product. I now show users a mock iPhone home screen that contains a clickable Spire app icon. After users “open the app” within the prototype, I guide them to the screen(s) I want to test. When users understand what they’re looking at, their feedback is far more valuable.
8. Don’t box yourself into wireframes.
Every great UX crash course or portfolio piece implies that wireframes are the “correct” design fidelity for early testing because colors, pictures, animation, and copy are distracting. But when I applied this to my design of a music video recording feature, users didn’t realize that the boxes with silhouettes were “videos” or that the colorless waveforms were different “audio” tracks.
To get your idea across to users, design at the minimum fidelity at which the idea will be generally understood without your commentary. In my video design, I needed to use photos, colored audio tracks, and basic animation in order to convey the concept, elicit a natural reaction, and get useful feedback.
7. “Imagine” as little as possible.
A common practice in user research is to create a “scenario” that puts the user into a certain mindset. When I first began testing a new feature, I told users to “Imagine that you’re about to film yourself playing guitar in your bedroom.” But users who didn’t play guitar or record in their bedroom couldn’t relate.
To get realistic feedback from a scenario, ground it in each user’s reality. I started asking upfront: “What do you play? Where do you record? Do you write original music?” Then, I inserted these into the scenario: “Let’s say you’re filming an [original / cover] while playing [your instrument] in [your usual recording location].” The feedback I received became far more specific.
6. Have a conversation, not an interrogation.
Before my first user sessions, I wrote an extensive script of questions ordered by topic. It helped to remind me of what I wanted to discuss, but distracted me from actually listening to users. I was usually thinking about what to ask next. The result was a one-sided interrogation instead of a two-sided conversation.
To get users to share feedback openly, make a list of things you want to learn instead of a script. After testing a product repeatedly, the questions will come naturally. Allow the user (and yourself) to go on short tangents. Natural conversation leads to them liking you more, which leads to comfort, which leads to sharing, which leads to insights you never thought to ask about.
5. Embrace the awkward silence.
In conversation, we seek to avoid silence at all cost. Especially when we’re talking to customers who we want to make as comfortable as possible. In my first few sessions, my attempts to immediately fill gaps in conversation led me to interrupt users and stifle their train of thought.
To keep users talking, wait a few seconds after they finish before saying anything. They want to avoid awkwardness just as much as you do, so they’ll likely say something else. If not, it gives them more time to think about the topic or task at hand. This tactic has led users to talk candidly for 20+ minutes straight with just a few words from me here and there to guide them.
4. Stay away from your team.
When I began running remote sessions, a fellow designer sat in the room with me off-camera and took notes. As we promoted the sessions internally, PMs and devs started showing up. Eventually, there were 6–8 people sitting around me at most sessions. It was great that non-designers were getting involved, but distracted me with whispers, typing noises, and a stuffy room.
To enable whoever wants to view your sessions to do so without throwing you off your game, book two rooms: a moderator room for you and 1–2 key team members, and a viewing room for everyone else. This way, you can run your session with focus while your larger team can talk to their hearts’ content. I set up a chatroom so that they can communicate with me during the session.
3. Make yourself useful to non-designers.
The phrase “user research” limits our scope of conversation to design and potential improvements to a product. In reality, we are often speaking with customers who have gone through an entire journey between learning about our product or company and present day. This makes for a great opportunity.
To make the most of user sessions, re-frame them as “customer conversations” and ask your co-workers what they want to learn. Our marketing team was interested in which ads led users to our website. Our customer care team wanted to know whether users found our online tutorials. Once I started asking questions for my teammates, user research became more popular.
2. Sum up and share findings NOW.
Before my first user tests, I patted myself on the back for creating a great file structure for scripts, notes, and video/audio recordings. Despite this, the sessions started to blur together. As information piled up, it became buried in folders instead of visible and actionable to the team.
To translate user research into product improvements, send a summary of each session to your designers/devs/PMs right away. Creating timestamped notes and video highlights in Lookback has enabled me to share critical findings and clips within 30 minutes, spark conversations that lead to immediate decisions, and easily go back to specific moments months later.
1. Always Be Testing (ABT).
I was at a Boston design Meetup, AKA happy hour at a bar, when I met a man who loved recording music in his spare time and often posted covers on YouTube. I told him that I was working on designs for a music video recording feature, he said to keep him posted about it, and the conversation moved on.
The next day, I told my team’s user research guru that I met a potential user. He said, “Did you show him the prototype?”
I replied, “No…should I have?”
He shook his head and said, “Dude…Always Be Testing.”
