7 things I learned from a Summer of Guerrilla Usability Testing for a Startup

Guerrilla Usability Testing, big words there. Let’s break it down!

A quick google search for ‘guerilla’ reveals the following:

These definitions seem to make the term ‘guerilla’ intimidating. Let’s just hone in on the ‘activities performed in an impromptu way’ for the purpose of our understanding and it would be more obvious when we get to the ‘usability testing’ part.

Usability Testing is the process of watching people use a product be it an app or a website and probing them about why they did what they did, how the felt and what could be done to make the experience better.

Yeah, that’s me and I spent a lot of time white-boarding!

So, to put things in perspective, I was a User Experience Intern at JOOL Health, an Ann Arbor based health and wellbeing startup, last summer. Along with a fellow intern, my job was to make quick prototypes of the mobile app that we were building to help people lead a healthier and purposeful life and get informal feedback about the app by showing the prototypes to people who were hanging out in coffee shops and around campus.

How did we do it and what did I learn?

  1. Feel the fear and do it anyway

To go to a coffee shop, interrupt unassuming people in the midst of the tasks they were engaged in, ask for their time and permission to review an app which they perhaps had no interest about seemed like an almost unreasonable and daunting exercise at the beginning. Yet, we decided to bite the bullet and do it anyway. Sometimes the best way to overcome your fear of doing something is actually to take the plunge and do it!

2. Hunt in pairs

My task was made surmountable and less unnerving by the fact that I had a wonderful and cheerful colleague who was willing to take the lead in talking to people and assuage me that it was doable. Most usability tests usually need a team of two to run it smoothly: there is an interviewer who sets up the stage for the conversation with a usability test participant and a facilitator who takes notes, make sure things stay on track. My colleague and I took turns when it came to the uncomfortable yet exciting task of approaching a stranger and asking if they had a few minutes to spare to help us out with the research.

3. Look for the lone wolves but remember to respect the territory

Initially, it was hard to determine who would be a good candidate at a coffee shop to go and ask questions. As we started doing these usability tests, we got better at identifying the prospects.

  • Don’t disturb those involved in deep conversations.
  • Look for the solitary reapers. Make sure they are not talking on the phone.
  • Look for the patrons with a casual demeanor and take your chances with them.

We weren’t always right in assuming that they would be willing to take part in the test, but we had our fair share of success.

4. Set the stage by planning ahead

Mugshot of a usability test plan

While we had little clue about the locations where and if we would find willing participants or how long we would have to go around the town looking to get the magical number of 5 participants suggested as a standard by Jakob Nielsen, there were certain things within our control. We created a test plan that clearly outlined:

  • The goals for the usability test
  • The questions were we looking to answer
  • A brief introduction about who we were, the organization we represented, the purpose of the test, what to expect and estimated time
  • Note to participants that it is the product that was being tested and not them
  • Instructions and tasks for the user
  • Questions to be asked before, during and after the usability test

5. Keep it short and directed

At first, we were unsure about how long a guerrilla usability test would take and how to prepare for it. Over a period of time as we conducted dozens of cafe intercepts, we got really good at estimating of whether a particular research or product question would be good candidate for a guerrilla usability test in terms of time requirement. 5 minutes was the sweet spot where you could get enough information without overwhelming the user but some of our tests ran up to 10 minutes.

6. Throw in a little topping to make the experience memorable

When we started doing these guerrilla usability tests, we just relied on the goodwill of our our participants and our persuasive abilities to convince them into doing one. Soon we discovered that there were limits to these and that we need to ask more questions that we could with just our sweet talking abilities. Enter incentives, $10 gift card at their favorite coffee joint. It helped us conduct slightly more involved and longer usability tests. But don’t just rely on incentives to carry you through.

7. Be Clear and Courteous

We were successful in doing the guerrilla usability tests because we were direct and clear in providing potential participants with the information about who we were and the purpose and intent of the usability tests. We were mindful that we were requesting for help. We were respectful in our approach and communication and thankful for a participant’s time. The exercise made me a better communicator and I’m now more confident of approaching a stranger and asking for help in evaluating a product.

Over the course of the summer of 2015 between May and August, my colleague and I had conducted over 50 hours of usability tests of which over half of them were guerrilla usability tests — cafe intercepts and usability tests with folks around the University of Michigan campus.

When we returned to the office after an afternoon or a day of conducting usability tests, my colleague and I were the most sought after people. The whole team would huddle around us and pepper us with questions like:

“What did you find?”
“Did the new version work?”

We did an informal share out and a more structured review of the findings and recommendations after reviewing the feedback from the usability tests. There was a strong correlation between findings from the guerrilla usability tests and the more formal and structured usability tests we conducted with employees at two different organizations. The findings and recommendations from these usability tests directly shaped the product design and the JOOL iOS app was launched in Oct 2015.

Later that year, I learnt about Google’s Design Sprint process and realized how the work I had done at JOOL Health had some parallels. At JOOL Health, we would come up with the research questions, sketch different options, prototype versions of different sections on the app, test with users and present findings and recommendations for taking the product forward. On occasions the entire sprint would be done in 2–3 days. Some of the more involved sections on the app took longer. Thanks to a design trek organized by the Design + Business club at the Ross School of Business, I was fortunate to meet Jake Knapp and John Zeratsky, Design Partners at Google Ventures later that fall and pick their brains about how they used the sprint process to help portfolio companies make design decisions.

In Closing

Conducting guerrilla usability tests was great baptism to the field of user research, in going beyond your comfort zone and in applying the skills learned in the classroom on a more practical setting. I was also making a career transition from being an online marketer to a UX designer and it was great to be a part of an early stage startup and help them out with questions around product design. If you are curious about the things I found out from the usability tests and how it impacted the JOOL app design, check out my portfolio.

Have you had experience with guerrilla usability tests? How did you go about doing them? What worked for you and what would you change? What suggestions do you have for conducting guerrilla usability tests? I’m eager to know. Share them and your feedback about this piece in your responses. Thanks!

Like what you read? Give Ram Kumarasubramanian a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.