The Demise of User Research?

“In a few short years, user research will no longer exist!”

I declared boldly — if, in retrospect, a bit riskily — during my job interview for Salesforce last year. Despite my prediction (or maybe because of it?), Salesforce hired me to lead user research for Salesforce’s CoreUX team. My blunt statement was not, of course, a repudiation of user research; I believe that user research is essential for any company to create great experiences for its customers and users. User research, is however, at a transitional moment, as fellow user researchers at other companies have also noted. While the user research discipline has always jockeyed for legitimacy in work cultures dominated by engineering and, more recently, by design, I think it’s currently encountering three transformative challenges.

(1) Knowing what people are doing: The rise of instrumentation

For decades, the only way that we could know what people were doing with our websites and software was to ask them or to observe them. We asked them to perform tasks in usability studies, and we observed them in their natural habitats. These days, we have more valid and reliable ways to know what people (and large numbers - “n’s” - of those people) are doing with our products. In large companies, people who use telemetry, log files, A/B testing to measure and interpret people’s product use have supplanted user researchers as the sole source of truth about how people use products. Seasoned user researchers know that analysis of large data sets of product usage data explains “what” people are doing (e.g., abandoning electronic shopping carts with items in them), but not “why” they are doing it. But, too often, user researchers stand by quietly while our stakeholders and internal customers learn to rely solely on dashboards and metrics, the kind of analysis that is typically provided by people other than user researchers.

(2) We all know our users: Democratization of access to users

For years, user researchers’ job descriptions were replete with phrases like “advocate for and represent the user.” The implication was that if it were not for user researchers’ tireless advocacy, developers and designers would create what they thought best with little awareness of users’ needs. Flash forward to the present, when people in non-research disciplines regularly speak for the user, and sometimes even to the user directly through research tools like UserZoom, UserTesting.com, or Validately (all of which have business models that capitalize on the assumptions that people conducting research have minimal user research training). And the connection to the user is great. Who wouldn’t want to work in an environment in which colleagues feel connected to users’ needs? However, when designers, product managers, developers, QA, support agents and what often feels like everyone in one’s company including the people in the payroll department claim to know users’ needs, it’s hard for user researchers to claim that they have a monopoly on knowing the user.

(3) No Time in the Sprint: Agile Development

Sidestepping the debates about what is proper agile methodology and how much one can legitimately deviate from the Agile Manifesto, the truth is that user research has never fit comfortably in agile approaches. Many a user researcher has grumbled about agile methods and how its emphasis on the iterative and the now leaves very little room for research practices that build a body of rich, valuable knowledge around users and their experiences. Occasionally, I chase down rumors of companies that have successfully integrated user research into their agile development process. More often than not, user research in those stories tends to be mono-faceted (evaluation of narrow features) and rarely includes formative or generative user research practices. Integrating user research into agile-like processes at large-scale enterprise companies — even at Salesforce, which is unparalleled in how it listens to its users and customers — is a challenge.

What Next?

Given the challenges currently posed to user research, how do I answer job candidates and graduate students about what they should do to succeed as a user researcher? I make two recommendations.

Quantitative Literacy

I say — somewhat dramatically — that I won’t even look at a job candidate that isn’t literate in quantitative methods. That doesn’t mean I think every user researcher should live in SPSS all day long or program in R. But I talk to many job candidates who dismiss quantitative methods as irrelevant to user research or who squirm uncomfortably with glazed eyes at the idea of doing anything other than an interview, field site observation or usability study. Properly trained user researchers choose their methods to match the research question, and, these days those methods are often quantitative because the data often resides in logs and databases. Executing a small-scale quantitative project, reading through some of the excellent books about quantitative research (I highly recommend Jeff Sauro’s books), taking an online (often free) course about data science are all ways to bolster one’s quantitative research skills. We need to be as comfortable with numbers as we are with words and behaviors.

The World of Strategy

I also urge user researchers to embrace and claim the world of strategy. User researchers are trained to focus on the tactics: how they can determine users’ pain points in a sign-up process or how salespeople close deals. They tend to stick close to the data and often way too close to a given product. However, the most effective user researchers I’ve seen stick close to the data, but then they use the data to do something incredibly important: talk confidently and convincingly about strategy and vision. Those researchers synthesize data across numerous forms and then inform not just product teams, but also executives and stakeholders in allied teams (e.g., Marketing, Product Strategy). It’s not easy, particularly for the empirically minded, to speculate, but, who better to inform strategic vision than the very people in a company whose entire purpose is to understand how people experience products — user researchers.

(Thank you to Ian Schoen and Kathy Baxter for their feedback on this blog.)

Follow us at @SalesforceUX.
Want to work with us? Contact uxcareers@salesforce.com
Check out the Salesforce Lightning Design System