Trouble Recruiting for Usability Testing? Try Immersion Research

Chris Kiess
NYC Design
7 min readSep 6, 2017

--

Let’s suppose you are designing a new app for a selected niche in a market. You’ve worked on the interface for the past month, prepared your specifications and even have a working build. Maybe it’s a recipe app for pastry chefs or a healthcare app for physicians or a client management app for a specific business segment. The one thing you are missing is users to test the app with. Since it is an app in a niche market, potential users who have domain knowledge to give you the right feedback are hard to come by. But, you know the market and the domain. So, you spend a week using the most recent build yourself. In the process, you discover a number of items you would consider low-hanging fruit and a number of other items only an expert user would find. This technique is commonly referred to as immersion research.

Immersion is a research methodology where you attempt to put yourself in the place of a user — using a product, piece of software or system just as the user would. You essentially become a user and do this for an extended period of time using a clear set of tasks you complete (just as a user would) and meticulously documenting your experience.

I once used immersion to evaluate e-texts and would actually use real reference questions from physicians and attempt to answer them using only the e-texts as a resource. I was working in a hospital at the time and routinely used text books for reference questions coming from the medical staff. So this approach using immersion allowed me to evaluate the navigation, the overall information architecture and the indexing of the e-books among other things. I became an actual user for the project and spent more than a month evaluating the product inside and out. It was an effective method given my lack of users at the time. Since then, there have been a number of systems, interfaces and products where I have used immersion to analyze them for lack of a decent user-base with which to test anything.

In the not too distant past, I immersed myself in my company’s hearing aid products and used myself as a test subject. I became an actual user, using the products as they were intended. But, there are a few caveats with this research methodology and it is not intended to replace true research with real users.

One of the primary problems with this approach is many designers are experts or near experts in usability analysis. This results in finding problems your average user probably would not find. This might not seem like a bad problem and it isn’t in some ways. But, designers tend to be more demanding than your average user and thus can come up with requirements and improvements to a product that may not be necessary or may simply add to the complexity.

When testing products using immersion, it is likely you are “an insider” in the sense that you belong to the company and know a lot of the “ins and outs” of the products users do not know. You are also likely to know what is and is not possible in terms of development. And, you probably know why products are designed as they are in many instances — the limitations and barriers that brought them to their current state.

Finally, though you can put yourself in the place of a user, you are not representative of the demographics or user base in any sense. You are simply not an average user and, in fact, are what some might term as, a “super user.” Designers have a tendency to look for the maximum ways to utilize a product often using 70–80 percent of the features in a software program — the features no one else uses. This, in a sense, makes the average UX designer a tough study and a very tough user.

For example, when I was evaluating e-texts, I found them to be severely limited in navigation. It just wasn’t easy to find the information I needed with the clunky search and navigation models. I tend to be something of a “search nerd.” So for me, this was a deal-breaker. But when I later conducted trials with actual hospital staff, I found they were not as demanding. Instead, they liked a product they could access right at their workstations. For them, the convenience of accessing a product without having to walk across the hospital to get to the medical library far outweighed any difficulty in navigating the interface.

None of the above caveats mean you shouldn’t use immersion as a research technique. You just need ensure you know the limitations and problems associated with such a methodology.

When I immersed myself in hearing aid products, I was not attempting to develop “rigorous research data” or to necessarily drive future design. I was immersing myself to, first, gain a sense of what the user experience was like for hearing aids. I have a significant hearing deficit and had worn hearing aids before. However, I had not worn them in years and never wore the brand of my current company. So, I was attempting to find my own likes and dislikes of the product as well as understand the frustrations our users experienced.

Another reason I placed myself in the role of the user for this project was the iPhone. My company at the time, GN Resound, was about to release a hearing aid that would connect directly to your iPhone. It was revolutionary and still is. I became a beta tester for these new hearing instruments and the iPhone app that controls them. I spent months wearing the hearing aids and using the iPhone app to control them and came across several shortcomings, a wish list of features that could be added and a general sense of the difficulties a user might encounter — all through the simple use of the application and native iPhone interface.

Have you ever used a product where you said to yourself: “Whoever designed this never used it!” I have used software in which there is no possible way the person who designed it ever truly used it for its intended purpose. What you are trying to avoid through immersion is being a person who designs for a product (or designs a product) he or she has never used.

Immersion, as a research technique, is a little easier with mobile applications than web or desktop applications because there are usually less features. But regardless of what I am evaluating, there are a couple of techniques I use in immersion.

Develop a checklist of the features and tasks the app allows you to complete as a user. You can use documentation to develop this list (if documentation for the app exists). Then work through this list in a real world environment, taking notes on each of the features and tasks. Those notes will include general observations, problems you have, bugs and the overall usability of the product. This is a good start in evaluating an app or interface. But, it only allows you to evaluate the features the design team came up with. What about features they didn’t think of? That is what my second technique is for.

Takes notes on your use of the application or interface as you use it in different situations. I will literally stop a task and quickly jot down a note when I get in a situation where a feature could be designed differently or there is something that could be added to improve the user experience. This doesn’t have to be a very formal process at all. I use a note taking app and just keep a list of problems and additions that aren’t covered through the checklist of features I developed. It’s a pretty simple process and you’ll be surprised at how much you will discover simply by using a product you are designing.

And that’s as difficult as it gets. No frameworks, interview questions, surveys or the like. It’s just as simple as using the product. I usually will use a product for a minimum of 4 weeks when using immersion as a research method. Longer is fine, but there is probably a law of diminishing returns at some point. This, of course, would depend on the complexity of the application.

There is one other caveat with this technique. You have to have enough domain knowledge to use the product. This probably is not a problem for most UX professionals working in e-commerce or for most app development projects. But, I consistently work in healthcare and there are not only access issues to using the software, but also issues in simply using the application from a knowledge perspective. For example, I could never play the role of a physician and use an electronic health record in practice. Even if I could, I wouldn’t have the domain knowledge to accurately find and represent problems within the system.

Here at Walgreen’s, I work on the pharmacy software for filling prescriptions. I don’t have the ability to actually fill a prescription or work through that process. But, I am a customer and faithfully use the mobile application along with the website to understand what is on the other side of my work. Though I don’t work on either the app or the website, I find being a customer helps me develop empathy and also an understanding of how system failures or downfalls in the pharmacy affect the customer.

In the sense that it is often hard to become the user due to domain knowledge, immersion is limited in this respect. But, there is still a wide field of applications where this technique can be used. It is a technique best used for consumer-facing applications in the early phases of development. And though it can save you time recruiting users, developing scripts, writing interview questions etc., it is not meant as a replacement for other types of research. Immersion gives you one point of data and any findings should eventually be validated via other research methodologies.

Immersion is a simple, but effective method. Try it out. Become a user and you’ll discover a good percentage of the problems and pain points a product has.

--

--

Chris Kiess
NYC Design

Healthcare User Experience Designer in the Greater Chicago area