Making New Mistakes Next Time: Running a UX Research Debrief
Two colleagues and I recently finished a 4-month project where we explored the ambiguous intersection of a couple of different concepts:
We conducted 30+ moderated feedback sessions and ran a survey with hundreds of participants, so we had loads of qualitative and quantitative data to make sense of. As always, it was fascinating to see some of our hypotheses confirmed and others upended.
Throughout the project, I played the dual roles of experience researcher and project manager. So when we emailed our final deliverable and called it a wrap, I organized our internal debrief meeting.
The premise was simple: dissect our process chronologically, peppering the conversation with questions and comments for ourselves and our process.
The big picture.
Talk about the details of the project as well as the people involved.
Be honest, not accusatory. Talk about your expectations and experience, and let others speak to theirs. Have a dialogue, not an argument.
Take good notes so you can refer back to your learnings when working on future projects.
Use the debrief as a time to reflect and grow, and learn from your experience so you can make new mistakes on next project, not repeat the ones you’ve already made.
Why did we have a debrief and why should you?
We accomplished an ambitious project and were proud of the work that we did. But it wasn’t perfect, because no project ever is. A debrief was the best way to get all of our ideas and feedback in one room and come to consensus on processes that we want to repeat next time and things we want to change.
While working in a consultancy, we have to address the project scope and methods in our work, but also other projects and initiatives that compete for our attention. At our firm, sometimes we are allocated at 100% on one project, but just as frequently we’re working at 2 at time, 50% each.
In consulting, debriefs are particularly helpful because we work with so many different people over the course of a year. It’s a time for whole group learnings, but done well, it can also reveal personal strengths and weaknesses that we can build from on future projects.
Who should be there?
Anyone who touched the project. Because we’re a consultancy, we include the business development person, the $$ person, the project lead, and anyone who worked on the project directly, namely researchers and designers. If they can give feedback about the process anywhere from scoping to deliverables, they should have the opportunity to participate.
When should you schedule it?
About a week after the project’s end. That way, you have enough distance so you aren’t still wrapped up in the emotion of the project, but close enough so it is all still fresh in your mind.
What should you talk about?
Everything from soup to nuts. Here’s what we covered in our latest project debrief:
- Scoping the project for future business development and resourcing
- Running the client and internal kickoffs to set client and collaborator expectations
- Designing the research methods to meet the Statement of Work and core research questions
- Conducting the research — recruiting participants, iterating on research protocol with client input, scheduling and other logistics
- Analyzing the data — using rigorous methods to interrogate and make the best use of the data we collected
- Delivering final materials that meet client expectations and needs, even if they don’t want to hear it (not all research findings are “good” news for the client, but honesty is what they hire us for!)
- Communicating throughout the project with each other and the client —clients we work with are our collaborators, none of this “swoop and poop” business
What questions should you ask?
In addition to the general topics to cover, here are some questions that every debrief can benefit from, consultancy or not.
- What would we do again?
- What would we do differently next time?
- How were we impacted as professionals?
- How were we impacted as a company?
- Where were pivotal points where we changed our approach?
- Did we push for the things that mattered, for the right reasons, and compromise where necessary?
While we can spend all day talking about survey questions and usability scripts, a debrief is also a perfect opportunity to talk about interpersonal issues. How did the team members collaborate with one another? Was communication fluid or static? Did that effect the project efficiency or ability to meet the timeline? Did the team grow together or apart?
While these latter questions are arguably the more challenging ones, they are what will help us grow as researchers in an increasingly collaborative world of work. These questions will address the roots of our working selves, insights that we’ll take with us no matter the project or company.