Feedback enables qualitative change.

Framing, Value, Causality, and the Non-Linearity of UX

Jay Morgan
Aug 10, 2018 · 7 min read

An argument on whether there is an “ROI of UX” arose on Twitter recently between Alan Cooper and Jared M. Spool among others. It covered whether there were grounds for asking a practitioner to state an ROI of UX; what it means for someone to ask a practitioner that question; and, one’s options in responding to the question. Their respective Twitter threads and Medium articles [Cooper’s “When Companies Question…”, Spool’s “Yes, Alan..”] provide calls-to-action for any motivated UXer, engineer, product manager, or leader to take up arms in an honorable fight. However, engaging the “what is the ROI of UX” question is fatally flawed because it is improperly framed in linear thinking.

Assessing the Question of Value

Mr. Cooper and Mr. Spool take orthogonal positions. Cooper asserts this question reflects that, if they’re asking you this, they don’t value you and/or UX, so you should leave. Mr. Spool rightly points out simply leaving a job assumes privilege most of us don’t have. Yet, he asserts two naively optimistic responses: One is that ROI of UX is easy to calculate, listing many ways in which UX generates value. While it easy to calculate ROI in cases of direct causation, this is refuted by the uncertainty of measures relative to the inputs, duration, and outcomes of UX. The second is that this question is a sign you are considered a candidate for leadership. I present the case when I was asked this. After citing points I’ve learned from Jared’s work, the response was a dismissive, “Yeah, but, we can’t be sure of that.” Clearly, the questioner is not always giving you a platform for leadership.

Mr. Cooper illuminated the problems with using an accounting-based frame like “return on investment” in a recent thread, [“ROI does not apply”]. I will work with an implied tenet of Mr. Cooper’s argument: The ROI frame is flawed because it is employed by people who do not do the proper diligence to understand UX applied to, present in, and producing outcomes for their business, market, or industry.

“What is the ROI of UX” is not a genuine question.

  1. A UX practitioner doesn’t have access to the costs, portfolio of investments, or revenues needed to calculate ROI.
  2. The business doesn’t effectively or properly measure UX activities or results as “investments”; and, therefore, isn’t able to assess any associated revenues with the respective UX-driven outcomes.
  3. If it were a genuine, knowledge-seeking effort, the asker would employ accounting, economics, or finance professionals to assess relevant costs, track outcomes, and associate resulting revenue changes to the UX activities and outcomes.
  4. It can also be demonstrated that they do not maintain a proper portfolio of competing investments by which ROI could effectively assess the value of UX.

If they did pursue this calculation rigorously with the requisite professionals, they would find that ROI is insufficient for measuring UX’s impact because it is a nonlinear practice. Both Cooper and Spool note there are many things UX does that cannot be tied to revenue, and that many of the outcomes UX contributes to or achieves are not directly monetizable. (I will not recite those here as their cases are sufficient.)

Our Non-Linear Reality

The “ROI of UX” frame is a case of looking through the wrong end of a telescope. ROI works when there is a direct causal relationship between cost, investment, and revenue amounts. ROI works as a linear function. UX is non-linear.

UX — if not our modern immersion in networked technology — implies that we can no longer presume the linearity of sender (company) to receiver (customer) as sufficient for doing business or providing services. Qualitative research teaches us that we can no longer presume heterogeneity of a customer, nor the conformity of customers to what businesses produce. To effectively frame and measure the value of UX, we must embrace the systemic causation of a non-linear practice, and orient business and operations around feedback loops of qualitative and quantitative data by which teams adjust their pursuits.

The practice of UX represents systemic causation [Lakoff, 2014] made up of the following 4 factors (illustrated with elements common to a commercial UX practice):

  • a network of direct causes — discovery research leads to personas and journey maps; journey maps inform designers of the end-to-end paths to successful outcomes; personas inform engineers of users’ behavior patterns, mind-sets, and desired outcomes.
  • feedback loops — ethnography feeds back into product concepts; conceptual research feeds back into early prototypes; card sorting feeds back into the information architecture of a coherent product; iterative testing feeds improvements and refinements back into designs as fidelity increases; usability testing feeds back into product improvement and maintenance; user interviews feed back into ongoing product development as it grows and scales.
  • multiple causes — unmet needs in a market; a new technology emerges; and, customers applying an existing capability to a new purpose.
  • probabilistic outcomes — the product will be used in different contexts by people with diverse — perhaps, adaptive — mental models, and across devices with different operating systems, the accurate outcomes of which cannot be feasibly forecast.

Looking through this telescope properly, we would see the need for and value in applying feedback loops (qualitative and quantitative research) to inform all facets of a modern organization to create (design and engineer) its products and/or services. This applies beyond commercial enterprises to social services, governments, utilities, healthcare, and education.

Reframing the Value of a Non-linear Practice

As Lakoff notes, “Reframing is not easy or simple… It doesn’t happen overnight. It is an ongoing process. It requires repetition and focus and dedication.” He encourages us to engage the other side in places where our values overlap. In an example of how to engage your politically-opposing relatives at Thanksgiving dinner, he illustrates this as asking them about times in their lives when they’ve experienced things that activate memories, feelings, and experiences in the moral overlap you share. [Lakoff, 2014]

We need to change frames from one that presumes direct causality between work — or, workers — and revenue to one that accounts for systemic causation, and accounts for a qualitative change in both practice and outcomes. Our goal as practitioners is to find the “moral overlap” with the organization’s members by engaging every facet of organizational operations in the feedback loops that enable sustained success. I propose the structure of this new frame consists of qualitative and quantitative research to inform the various parts of an organization with behavioral, cognitive, socioeconomic, and ethical factors it needs to design value for the humans it serves.

Let’s say you’re in a B2B software company. This reframing might begin as, “We’re seeing an increase in churn, while hiring more salespeople. Those are two costs chasing each other down a steep hill. Isn’t our goal to have satisfied customers? We need to understand why they’re leaving, and what they are missing from us, in order to satisfy them… How about this:

  • Our researcher interviews the churned clients about the deal-breaking things they experienced. This reveals their unmet needs, and where we’re not serving them as customers. Our cross-functional team observes these interviews, so the understanding is absorbed across the business, and we can process it from all perspectives.
  • Then, we bring some of the churned clients into participatory design workshops. The cross-functional team is partnered with the clients, so we devote all aspects of the business to understanding and meeting their needs. We’ll be able to model needs — maybe, users — we’re clearly missing; and, find the places and ways we can support them, even delight them.
  • From there, we move into iterative design. We’ll have represented all the parts of our business, who each understand customers, our business and technology uniquely, contribute along the way.
  • We keep working with the churned customers as we get the products from where they are now to where they need them to be. Having our cross-functional team participate along the way will prepare them for how our customers experience our products, and educate them about what it takes to get from user needs to a product in market.

…This is a fixed cost for the period of that engagement — say, a few months start-to-finish, as opposed to ongoing costs of salaries and bonuses for salespeople, who are pushing a product that previous customers found lacking. The outcomes could (1) earn back some churned customers; (2) benefit other customers by generating unrecognized value that’s relatively easy for us to deliver; (3) help us build case studies that would make sales easier; and, (4) demonstrate our good will as a company that serves its clients. It will also create relationships across the business in places that will drive cross-functional understanding and communication that will enable us in the future.”

Towards a New Model

I encourage us to embrace and state the values of practicing and enabling UX as a qualitatively different practice from business as usual. Stand proudly for the principles and methods we advocate as a UX practitioner, and be clear about how they translate to value for our organizations. As Lakoff tells us, “the truth alone will not set you free. Just speaking knowledge to power doesn’t work. You need to frame the truths effectively from your perspective.” [2014] Be clear and firm that this is about systemic change, that the value and outcomes are greater when qualitative feedback informs every facet of the organization.

There is raw power in reframing how business is done in this networked era that is still becoming. The practice of UX promised monumental change, but, so far, it’s mostly been wielded for business by business to generate profit. By reframing how business is done we can change the way value is exchanged, wealth is distributed, and people participate in the economy. In order to properly achieve value today, we must apply qualitative feedback loops to inform each part of our modern organizations so that we create products and services that address the diverse behavioral, cognitive, socioeconomic, and ethical factors relevant to the humans we serve.

It is time to recognize and respect the complexity of our non-linear reality through the practice of UX today so that we can build the habits, value, and mind-set we need tomorrow.

Special thanks and respect to @danklyn, who helped me develop and hone this idea.

References:

Lakoff, George. (2014) Don’t Think of an Elephant, 2nd Ed. White River Junction, Vermont: Chelsea Green Publishing.

Jay Morgan

Written by

Aspiring cognitive activist.