Data Driven User Experience Research

Systematic and advanced analysis of complex data opens up new possibilities for increasing the power of UX Research analysis

Machine Learning and Data Analysis systems are powerful tools at the service of those organizations and professionals that can use them skillfully.

They can help narrow the gap between opinions and facts, analyze an impressive amount of the latter, find similarities and reveal connections, thus getting pieces of evidence otherwise hidden.

The design industry too, has seized the opportunities of these new technologies and begun to apply them to the production of products and services. Autodesk has been a pioneer of generative design, using machine learning and AI to generate design solutions and critically evaluate them. On the other hand, Airbnb demonstrated how visual analytics could automatically transform sketches and drafts into working interfaces and prototypes, showing us what could be part of the evolution of UI design in the coming decades.

But what if we applied these same techniques to experience and customer journey analysis, thus shifting the focus from design to research with users?

The experiences people can live along the Customer Journeys are many. The use of touchpoints, the sequence of steps, the motivations and needs that support them are multiple; not infinite, of course, but indeed many more than we can conceive in advance. Many of the relationship paths between consumers and companies are often overlooked or ignored — because they are not observed.

Companies, designers and researchers are hardly aware of this complexity, let alone its impact on business performance or design outputs.

We’ve been experimenting with this for a couple of years, and we’re using data science to enhance the heuristic depth of research activities with users. In addition, we intend to extensively use it on the entire customer journey, to map as many interaction events with the company touchpoints as possible.

The goal is the pervasiveness of the measurement: all the main interaction events between individual customers and the company’s touchpoints must be traced and analyzed to get a complete map of the people’s behavior and to be able to make assessments also related to business KPIs. The validity of the mapping depends on the availability of quality data and the feasibility of a calculation engine to process such information in a dynamic, flexible and complete way.

We are testing these practices with some clients, and have drawn some considerations.

Get real behavioral clusters and classes of experiences

The Machine Learning tools available today can group user stories by similarity, performing a “behavioral clustering” on the chains of events that characterize the sequence of interactions between users and the company, thus defining homogeneous groups of interactions and detecting behavioral patterns directly from the data.

Algorithms calculate a measure of “similarity” between the experiences — the “traces” that the customer leaves on the operating systems when interacting with a touchpoint. They consider the individual actions performed, the relevance of each event and their sequence within the users’ History, the channels of interaction, and other evaluations on the characteristics of the History (e.g. the temporal duration of the series of actions).

In short, it’s possible to detect groups of users who behave similarly and interact with corporate touchpoints in a similar way (e.g. they ask for the same types of information through similar channels): these are actual behavioral clusters linked to types of experiences.

There can be some surprises. For example, in how many ways do users go through a journey? Companies usually imagine a fair number of them: around ten. However, in a project done with an energy seller, we discovered more than nine hundred.

To handle this complexity, it is necessary to intervene with qualitative tools: grouping, looking for the causes of those behaviors, and for correlations with other factors.

We’ve realized that the data concerning the physical interfaces of the service are insufficient: we don’t have enough of them, and they are often poorly mapped. In addition, many of the behaviors tracked this way don’t make recognizable sense, and again, qualitative research makes up for the missing knowledge.

Understand the overall context

Not only behaviors and groups, but it is also possible to use the tools of data analysis and machine learning to study and understand the context in which the events we want to influence occur.

The entire universe of events is known: the paths, the actions, the segments of the customer base. It is therefore possible to make inferences and hypotheses about the motivations that generate certain behaviors or even the contexts that host them: demographics, use of specific tools for groups of people, problematic nodes.

Advanced tools can explain weak signals, anticipate emerging trends and thus address design actions with more confidence, without solely relying on the designers’ and researchers’ sensibilities.

Shape personas from data

The customer base segments identified can be the factual basis for constructing the Customer Personas of traditional UX Research: archetypal representations of the needs, aspirations, and behaviors of a particular segment of real users. However, unlike those built with the sole use of traditional research techniques, these ones rely on a massive amount of data. What’s more, thanks to these techniques, you get to build all the personas, linking each one to a particular segment of the customer base, as emerged from the mapping activities.

The personas constructed in this way can be the basis for further survey activities conducted using qualitative methods to explore the motivations, values and the emotional sphere linked to evaluating the experience.

There is also another relevant aspect: behaviors are transversal to the personas. Or rather, identical behaviors can be acted out by different people in terms of demographics or type of spending.

We have learned that making a one-to-one correlation between personas and behavioral clusters is erroneous. Thus a second-level characterization is necessary, conducted with the sensitivity of traditional qualitative research.

For example, the behavioral cluster that we named “The Complainers” because of their constant recourse to the complaint service of one of our clients, contained within it an assortment of people animated by very different motivations. If we had traced them back to a single person, like “Pietro — The Grumpy”, we would have lost an impressive amount of information.

Big data + fast data + small data = a new approach to user research

The traditional qualitative-quantitative approach allows us to evaluate the quality of the experience offered by a brand to its end customers and interpret their behavior and motivations. But — again, this is its significant limitation — it only succeeds in doing so on a few Customer Journeys among all those that occur. The advanced techniques of data analysis applied to Customer Experience allow us to discover and map them all — but not to evaluate them — and then consider them in their concreteness. But all customer-company interaction processes have blind spots that data cannot detect: this is the limit of the data-driven method.

The innovation of the combined approach between traditional research dealing with small data and data-driven investigation on big and fast data, lets us overcome the limitations of both methods, allowing us to monitor a large number of people and to verticalize the studies on focused and limited areas. Furthermore, the data suggest the research topic to deepen: the most relevant for the business or the most ambiguous. From there, it is possible to build more complex operational KPIs and establish a comprehensive monitoring and control process. The next step is to infer correlation or even causality with the main KPIs of interest to the business from the operational KPIs.

The KPIs identified by this new hybrid method of analysis allow a continuous testing of the experience’s quality while identifying and implementing targeted, measurable, and practical improvement initiatives.

Combining these two analysis approaches allows us to take the best of both worlds: to have full awareness of all Customer Journeys, to eliminate the dimension of arbitrariness linked to qualitative methods, to exclusively intervene on actual journeys, well mapped in all their points, and then develop design actions starting from a factual basis.