Conversational UX: Repair Sequences

1. Current Problems in Conversational UX Design

The last six months have been a pretty busy time in the Conversational UX (CUX) scene in Germany and the UK. I’ve taken part in lots of exciting courses and events related to conversational interfaces and have had the chance to speak with dozens of colleagues from across the world about recent developments in the field.

As a CUX researcher myself, I usually teach and speak about the application of linguistics in the design of conversational interfaces, focusing on my areas of specialization in Conversation Analysis (CA), Pragmatics, and Ethnography of Communication (in another post, I’ve talked about the huge value CA brings to the table as a source of knowledge about natural conversation). But what makes these events really come alive for me is the ongoing direct interaction with the public and other researchers interested in CUX.

Neon Gold Innovations, Club Häkken, Hamburg Reeperbahn, and Internet of Voice

This interchange and exchange of ideas has given me some valuable insights into the current state of the conversational interface industry in Europe:

  • Companies have realized that the work of conceptualizing human-to-machine (H/M) conversations can only be done by professionals with a good knowledge of natural language. More precisely, they are looking for CUX designers who can create natural conversations that are as close as possible to human-to-human (H/H) communication. This is the ultimate goal: creating conversational interfaces that are natural interfaces.
  • There’s currently a knowledge gap that needs to be bridged. The linguistic understanding of natural conversation developed by researchers working on CA, Pragmatics, and Sociolinguistics needs to be transferred across to, and integrated with, the field of UX. This is a problem I specialize in. My approach is to develop and apply solutions based on the theory and methodology of CA and then feed these solutions into the creation of a scientific framework for CUX design. These are the kinds of solutions I teach on my courses and speak about at CUX events.
In the UI/UX Designers Berlin Meetup I found my target audience and meet new friends
  • The iteration of conversational interfaces is not yet pursued in a way that makes proper use of extant scientific principles and results. In order to ensure that future development is grounded in sound experimental methodology, UX designers need first-hand information about how linguists and anthropologists develop their experiments. This will give them a framework for analyzing natural conversations using both qualitative and quantitative methodologies.
  • Tools for prototyping also need to improve. The main problem here is that there is still no application that is able to capture the textual and interactive nature of conversation, and then integrate it properly into the well-established design and thought processes of UX designers.

2. UX Professionals and Linguistics

As far as the transfer of knowledge from CA to UX is concerned, I’ve found that there’s a huge appetite for this among UX professionals. When I prepare my courses and talks, I know that something like 85% of the audience are going to come from a professional UX background. That means framing my presentation around a critical question:

How can I teach linguistic terms in an accessible and comprehensible way to UX designers?

The reason this question is critical is simply that

Technical terminology from linguistics is too obscure for people not trained in the discipline.

That doesn’t mean we have to “dumb down” the topic, though. Instead, we need to find ways in that don’t rely on detailed background technical knowledge. Fortunately, everybody uses conversation in their daily lives and UX designers are no exception. We all have an “instinctive grasp” of how verbal communication works. This intuitive “feeling” about how language fits together is more sophisticated in big cities like Berlin, where the vast majority of professionals are also multilingual. Living in a multilingual environment means that most UX designers already have built-in metacognitive abilities for cross-language analysis.

Sometimes I also speak about feminism, AI, and conversational interfaces for the women in tech community.

As a consequence, when training UX designers in conversational design, it’s possible to use the “basic grasp” they have about conversation as a starting point from which they can easily be introduced to complex linguistic concepts and terminologies.

Based on my experiences in training sessions, UX writers and copy writers are already in an advantageous position for acquiring further linguistic concepts and applying them effectively in conversational design. The structural sympathies between the fields make me very curious as to how well somebody with a degree in Linguistics will do in conversational design once trained in the interactive limitations imposed by the technology.

2.1 A Methodology for Teaching Conversational UX

After carefully analyzing the feedback received in each public talk and course, I decided to create a methodology for teaching CUX Design that transfers the interactive protocols used in H/H conversation from CA to CUX. These protocols can then be used in the design of H/M conversation in a CUX context. I present conversational protocols, elements, mechanisms, and strategies in a comprehensible and accessible way that anyone can understand, regardless of their training or professional experience.

The methodology I follow is simple and effective. For each conversational element belonging to the framework of natural conversation:

  1. I introduce its technical name and definition as understood in CA. This enables the designer to identify its place in the general framework of conversation and conceptualize it more effectively and scientifically.
  2. I illustrate with examples from real H/H conversations how the conversational element works, as well its main structural, cognitive, and interactive characteristics. I also describe in detail the conversational structures it generates or is integrated into.
  3. I provide a very clear example of how to exploit the defined conversational element when designing an H/M conversation. This is an agnostic solution for CUX design that has the strategic advantage of being functional across diverse types of design platforms and conversational interfaces (both text- and voice-based).
  4. I propose a set of collaborative exercises among peers that scaffold the concepts and enable deep assimilation.
  5. I provide the bibliographical references for studies of the concept, so the attendee can research the topic further if they choose to do so.
Method is just everything.

3. A CUX Framework for Repair Sequences

The next step after thoroughly testing this methodology for teaching CUX design is to write a book about CUX design for error recovery solutions in H/M conversation. I began work on this phase in October and the first (Spanish) version of the text will be ready to come out in January 2019.

But I still need help from the CUX community. Above all, I need to check if the way that I’m introducing concepts from CA is the most appropriate for the “general public” interested in the topic (whatever that may mean in the case of an emergent discipline like CUX). It’s for this reason that I want to show in this post some of the content I’m working on and if it is comprehensible to UX designers. I also want to share a chapter of the book with the community before its release, so I can get feedback from you and improve the quality of the final product.

So, what I want to do next is illustrate in an understandable way the methodology for teaching CUX design I use in my book on error recovery and repair sequences in H/M conversation.

By the way, the working title of the English version of the book is:

Conversational UX: Repair Sequences
I need help from you!

3.1 Terminology: Error, Repair, and Repair Sequences

Error

An error is an illogical behavior that needs to be repaired in order for the conversation to continue. Contrary to what is commonly thought, errors constitute a significant part of H/H conversation. For instance, in the variety of Spanish spoken on the Iberian Peninsula, 13% of the sequences we use in the topical core of a conversation are represented by repair exchanges.

In H/M conversations, errors are commonly seen as representing misclassifications committed by the lowest acoustical attributes of the base technology. In fact, they occur throughout the layers of perception, cognition, and mutual negotiation that comprise an intelligent conversation. They are also generated by both by the system and by user behavior.

Repair

A repair is the action of fixing an error, so the conversation can get back on track.

When humans apply repair practices, we don’t consciously identify them as mechanisms for recovering from errors. In other words, humans don’t think about the fact that we’re fixing an understanding problem when we say something like “Sorry, I don’t get what you mean”, just as we don’t tend to meta-classify the expression “hello” as a greeting when we use it with our neighbor.

This has a useful consequence: if we extract repair practices from natural conversation and use them in H/M conversation, users will most likely not realize that they have been in an error state. Or, at least, it is natural to be in an error state in that way for them.

Repair Sequence

A repair sequence is the group of intents exchanged between the listener and the speaker that solves the interactional problem and brings interlocutors back to the topic they were discussing.

Wrong ways always make you turn around and return to the main lane.

3.2 Cognitive and Linguistic Features of Repair Sequences

Repair practices in natural conversation are based on two main human cognitive behaviors:

  1. Humans repair interactive problems only if they seriously interfere with the development of the ongoing conversation. The unique condition under which humans cancel the ongoing topic they are discussing and start a repair sequence is if they face a context of complete interruption of mutual understanding.
  2. Interactive problems related to hearing, speech, or understanding represent one-time interruptions inside the conversation. This means that humans tend to solve errors locally and with the minimal number of intents possible, because their main interest is to return to the topic they cancelled before the repair started.

This last point is the main cognitive reason that dialog design implementations of error escalation strategies do not work for users. This design technique — inherited from previous practices for IVR systems and sometimes called progressive prompting — forces the user to give more detailed instructions or examples thought more of four intents, surpassing the amount of intents which a human can cognitively tolerate in a context of repair. We can see this in the following extract from an H/M conversation.

As we can see in this exchange, the escalating error technique does not take the interlocutors out of the error state in a natural way. On the contrary, the interaction leads to a complete interruption of the conversation, because the best option for the user at this point is to start everything from the beginning again once he knows how the system collects information from him. In other words, in order to properly communicate with the system, the human has been forced to learn the unnatural conversational practices of the machine.

CUX today works from a completely different philosophical, cognitive, and interactive start-point: machines have to learn human models of conversation so they can properly interact with us.

3.3 The Structure of Repair Sequences

In natural conversation, repair sequences comprise a minimal structure of three turns or intents. These are called ‘origin of the problem’, ‘repair initiation’, and ‘repair outcome’.

Origin of the problem

This intent is also sometimes called the trouble source or reparable. It contains a hearing, speech, or understanding problem.

This intent must not be confused with the base of the problem, which represents the physical or cognitive cause of the error (a background noise, some lack of attention, a lapsus linguae, a mispronounced word, etc.).

Repair initiation

This is the turn in which one of the interlocutors focalizes attention on the presence of a communicative breakdown. Humans do this by using the technique of framing, that is, by articulating a question or repeating the previous intent in whole or in part.

Repair Outcome

This is the intent in which the solution or the abandonment of the problem is materialized. In the latter case, it is critical to mention that humans often simply ignore the problem if its solution entails too much cognitive effort. The reason for this is that repair sequences that are too extensive steal the attention away from the main topic under discussion. They thus threaten the coherence of the conversation and its obligatory topical continuation.

3.4 An Example of a Repair Sequence from a Natural Conversation

The next example illustrates the minimal structure of a repair sequence extracted from a natural conversation (originally in Spanish).

3.5 An Example of the Same Repair Sequence in a Human-to-Machine Conversation

In the next example, we can see the subtype of repair introduced in the previous section implemented for an H/M conversation. To be precise, what we have here is a case of other-initiated other-repair in which the machine holds the initiative for the repair initiation.

4. Help me to Write the Best Book about CUX and Repair!

If the material I’ve shared in this post about repair in H/H and H/M conversations makes sense to you, enriches your knowledge and understanding of natural conversation, and gives you more resources for your CUX design, then please give me your feedback and help me to write the best book possible about CUX and repair sequences.

Here’s the deal:

  • The book will be published in two languages, Spanish and English. This means you can request the sample chapter in your preferred language.
  • I provide the sample chapter of the book in exchange for personalized feedback. I’ll have a 30 minute face-to-face interview with you over video conference in which you’ll share your ideas (both positive and negative) about the material.
You can already request your copy of the chapter here! :)

The idea is to improve the final quality of the text by taking account of all the feedback received. This is how the content of the book looks right now for the Spanish version:

To Summarize

Disciplines and new areas of expertise do not emerge disconnected from the natural ecosystem that nurtured them. In the case of CUX, one of the most acute problems to be solved is the transference of linguistic knowledge about natural conversation from CA to the design practices of UX. But we need to create solutions using the language spoken by the audience. At the same time, we also need to build a community around our new profession.

CUX designers and researchers of the future, help me pursue these goals! And if you think you have other friends who might also want to get involved with CUX, please share the post and the contact form!

References

Martinez Carrillo, M. C. (to appear in 2019). Conversational UX: Repair Sequences. Most likely to be published independently at Amazon.