How User Researchers Can Adapt Conjoint Analysis

As UX design and research continue to become integral for business decisions and product development, we need to broaden and curate our methods for gaining actionable and quantifiable feedback from users. We want realistic data that informs product management, sales, and executive leadership on the right decisions for products and services.

Typically used for market research, conjoint analysis employs the more realistic context of asking respondents to evaluate potential product profiles (sawtoothsoftware.com). As user experience researchers, we can expand this definition to include the evaluation of service, experience, and design profiles in addition to product profiles. Instead of allowing respondents to isolate and cherry-pick specific features or attributes they find appealing, choice-based conjoint analysis asks the respondent to select an entire scenario which will include a balance of desirable and less desirable attributes.

When I first discovered conjoint analysis I found it to be relatively dense. There are several articles and YouTube walk-throughs to teach you traditional conjoint analysis. Tools like SurveyGizmo and Excel plug-ins can enable you to do proper conjoint analysis studies. However, as a user researcher, I find we often don’t have the time, readily available user group, or statistical experience to conduct a fully-fledged conjoint study. This article will walk through 3 key ways to adapt methodologies within conjoint analysis for a fast-paced user research environment.

THINK IN SCENARIOS, NOT FEATURES. Let’s walk through an example of a conjoint analysis study looking at tourism options for an upcoming trip. The respondent is presented with 4 options: a trip to San Francisco, D.C., Las Vegas, or the choice of not going on vacation at all.

Quick tip: it’s important to provide the user with the option to not choose from any of the presented scenarios, as this can be a very informative insight about what product, service, or experience you are offering.

Source: http://www.sawtoothsoftware.com/products/conjoint-choice-analysis/conjoint-analysis-software

We can see that within each option, there are variations of the same attributes listed on the left-hand side. These include destination, the number of nights, accommodation, hotel type, car rental and price (per person).

In true market research, there may be between 12 and 30 conjoint scenario prompts questions provided to each user. These questions would cover all possible variations by multiplying the number of attributes by the number of variations. For our purposes in user experience research, we may only have time to present one well-crafted conjoint analysis question that gets at the heart of our assessment. We want to understand where our users place value based on holistic, trade-off scenarios rather than giving them a list of features and allowing them to craft an ideal offering on their own. Essentially, we know people want it faster, better, and cheaper. But this won’t help inform product managers and developers who are trying to ascertain which elements of an offering are critical versus those that users are willing to do without.

Quick note: in true conjoint analysis where a user is presented with 12 to 30 scenarios to choose from, each feature will have a part-worth utility score that assesses how much weight that particular feature has in terms of a user’s scenario selection. Essentially, this part worth utility assigns a percentage of importance on a scale of 0 to 100 in terms of a user making a decision (e.g. price has a part worth utility of 80% meaning it’s a very important consideration when choosing between scenarios). With user research, we need to bring in our qualitative knowledge and understanding of the space to assess between 2 or 3 presented scenarios and get a holistic scenario score, e.g. 75% of users selected the Washington D.C. trip.

We want to move away from asking users about features or elements in isolation, and rather frame options within a scenario or holistic context. This will better inform the meaning behind qualitative user quotes and help our team members make the tough calls that product design requires.

KNOW YOUR CONTEXT…OR LINK UP WITH SOMEONE WHO DOES. Do we want to understand the implication of a product’s price within the context of additional storage, a new UI and a release date? Or do we want to understand the implication of a product’s price within the context of collaborative tools, integration with open source libraries, and amount of employee training required for use? Often as user researchers, we are asked to simply “learn everything” about an area or “go talk to these people.” By framing the research conversation around an adaptation of conjoint analysis, you can create an artifact for your product managers and developers to rally around with you. You can decide as a team what the proper recipe should be for crafting comparative scenarios and feel confident going into user engagements that your results are going to be of value to the larger team.

LEVERAGE USER RESEARCH METHODS FOR INTERNAL ALIGNMENT. When I first started learning about conjoint analysis and the ways I could adapt it as a user researcher, I saw it as a great tool for product/experience offerings. However, I feel it can also be a powerful tool for internal alignment and expectation management within teams. In the same way that we’ve found the futility of users wanting things faster, cheaper, and smarter, our greater product teams want us to talk to more users in less time and get the best insights. While we always strive for the best process and results, we can leverage the same trade-off methodology in conjoint analysis for internal decisions as well as external product decisions.

For example, we can present our team lead or product manager with the scenarios below at the onset of a project research initiative.

Hopefully this article will help you leverage some underlying methodologies within conjoint analysis in your user experience research and design work! Please see the links below if you want to conduct a fully-fledged conjoint analysis study.

All thoughts expressed are my own. http://www.gabriellacampagna.com/