Hands-on practice, although important, does not necessarily lead to expertise. The best user researchers analyse their work, deliberately and consciously. By reflecting on a user research activity, they are able to increase the learning from a situation, identify their personal and professional strengths and find areas for improvement and training.
User research is an interesting blend of both theory and practice. Practitioners need to be aware of theory to avoid making fundamental mistakes, such as biased protocols, that make it impossible to interpret the results of research. And practitioners also need experience to decide on a research approach that is “good enough” to meet the research objectives. Absence of practical experience can result in a user researcher proposing activities that don’t fit the project’s budget or the organisation’s business goals.
So it’s no surprise that most practitioners believe that the route to getting better at their job is to combine day-to-day experience (“learning on the job”) with skills-based training courses to fill the gaps in their theoretical knowledge.
As someone that has spent 20 years training and coaching user researchers, I obviously have some sympathy with this approach. But I’m also aware of its limitations. Although some of the people I work with go on to great success, both as accomplished user researchers and as leaders of design teams, many others have a middling career in user research or swap to a different career path.
As I looked at the differences between the groups, there was one behaviour that stood out. The most successful user researchers didn’t just attend training courses and do the work day-to-day. They consciously and deliberately reflected on their work.
What does it mean to be a reflective user researcher?
It’s common for user researchers to log the activities that they carry out either in a notebook or using electronic tools. For example, some user researchers maintain a spreadsheet that lists details such as date, the type of research and a short description of the activity. One example of this can be found in Amanda Stockwell’s article, Never Trust a Skinny Chef. Every user researcher should be doing this, but this isn’t the same as being a reflective user researcher.
And simply saying that a user research activity went well or poorly is not being a reflective user researcher either.
Being a reflective user researcher means that you think critically about the work you have carried out. This is different from being critical of what you have done: it’s about analysing your performance. Why did you do it that way? What theory underpins your approach? What organisational constraints did you need to work within? Are there alternative methods you could have used?
The reason reflection is so powerful is because experience alone does not always lead to learning. To really learn from an experience we need to analyse it, deliberately and consciously.
When I’ve asked the more successful user researchers why they engage in reflection, I hear different reasons from different people. Some of the reasons are:
- “To improve my practice.”
- “To identify training gaps.”
- “To create a portfolio entry.”
- “To decide what I should do, and what I shouldn’t do, next time in the same situation.”
- “To see if there are general research ‘patterns’ I can apply to situations like this in the future.”
- “To create talks for conferences.”
- “To create articles for web sites.”
- “To run ‘what if…’ scenarios”. (For example, you may have wanted to do a field visit but been prevented from doing so because of budget constraints. On reflection, how would this have helped or hindered the final impact of the research?)
What form does reflection take?
Reflection can take many forms and it’s important to choose a method that suits your way of working. For example, if you spend an hour commuting every day, you could reflect in your head: it’s not critical to have it written down. The form of reflection is less important than the way you reflect. Superficial reflection, or doing it because you feel you have to, won’t get you very far. In the best kind of reflection, the user researcher analyses what they have done, using evidence to decide how the activity could have been improved.
Nevertheless, a written record does have some benefits: the act of revisiting your journal on a periodic basis reminds you about what you thought was important. Think of the way Facebook reposts an event from your past. It immediately reminds you of something that you deemed important at an earlier point in your life. As you become more experienced, revisiting the event gives you the opportunity to think of advice you would give to your earlier self.
Here are some ways to make reflection part of your routine:
- Keep a Moleskine log book.
- Create a notebook in Evernote.
- Build a template in Word.
- Create a video or audio diary.
- Discuss the research activity with other user researchers in your organisation.
- Seek feedback from people on the project team whose opinion you respect.
- Ask for feedback from your research participants.
- Review participant videos (if you have them) to identify good and poor practice in the way you moderate usability tests or run user interviews.
- Talk with a mentor or a colleague outside the team or organisation where you work.
Another approach is to hold a project post-mortem or a sprint retrospective. Although this is good practice at the project level, this method isn’t an ideal way to improve the user research you carry out. This is because the attendees will want to discuss issues other than user research, so you may find the user research component is discussed for only a few minutes. Also, the team may not be that interested in user research or be able to provide the quality of critical insight that you need.
When should you reflect?
This is something you should be doing continuously: not necessarily at the end of a project, but after each stage of a project. For example, you might do it after you’ve scoped the research plan with stakeholders, after you’ve recruited users, after running a particularly good (or bad) user session, or after a show and tell with the design team. As a rule of thumb, whenever you find yourself worrying over a phase of the work, or feeling good about how something went, that’s probably a good time to reflect. Initially, to make it part of your routine, try reflecting at the end of each sprint.
A format for reflection
If you’re struggling for more specifics, here’s a format you could use. But remember, it’s important to do this in a way that will work for you, even if that’s simply having a ‘meeting with yourself’ in a coffee shop.
- What user research activity did you carry out? Include the date, the type of research, a short description of the activity, the sample size, the session length, and the total time spent.
- Why did you do that specific user research activity rather than something else?
- What, specifically, went well? Try to identify 2–3 aspects.
- What, specifically, went badly? Try to identify 2–3 aspects.
- Analyse. For each of the aspects you have identified as good or bad, ask “Why?” Be careful to avoid superficial reflection and aim for a deeper level of understanding. Consider using the “5 Whys” technique to get a fuller understanding.
- Integrate into your practice. Thinking about what went well, how can you apply it to other situations? Under what situations might this approach not be effective?
- Ask, “What if?” Thinking about what went badly, how can you prevent this from happening in the future? What would you do differently in this type of situation next time?
Use the format above to reflect on your most recent user research activity. What would you have done differently?
Thanks to Philip Hodgson and Todd Zazelenchuk for comments on this article.
Why not join the thousands of other people taking my free online user experience lessons?
Originally published at userfocus.co.uk.