Conversational UX: Knowledge and Conversation

Machine, I don’t understand you

Tacit knowledge, Intention and Order in Conversation

In an earlier post, we introduced Conversation Analysis (CA) as the sociolinguistic discipline concerned with the systematic study of what is known, in general terms, as interactive talk or talk-in-interaction and, more explicitly, everyday or spontaneous conversation.

CA approaches the study of the conversation from three interconnected theoretical assumptions:

  1. The core importance of tacit knowledge processes in the creation of meaning during interaction.
  2. The consideration of utterances as actions.
  3. The principle that conversation is sequentially organized.

These three concepts guide the researcher across understanding conversation. In the field of human-to-machine interaction, they can also be adopted by VUI designers as theoretical frameworks for the construction of more natural and advanced dialog systems as a way of interacting with intelligent assistants and speech-enabled devices.

Starting with a series of three posts, in which we will analyze each of the previous concepts individually, next, we describe the benefits of using a dialog writing methodology for VUIs based on the recognition and application of the tacit knowledge procedures on which conversation cognitively is rooted.

Tacit knowledge processes in conversation

CA considers that interlocutors create meaningful conversations by explicitly codifying in them cognitive tacit knowledge processes, which ensure the proper production of natural interactive talk by following an explicit order.

The ultimate aim of the discipline is to explain the nature of the rational knowledge that the interlocutor needs to apply, together with a set of sociolinguistic skills, in order to:

  • Provide meaning to the utterances of the other speaker.
  • Adapt the content of each utterance to the current state of the ongoing interaction.

The basic working hypothesis in CA is that conversation constitutes the verbal materialization of language as an instrument of thought. Since the ultimate goal of language is communication, it is logical to assume that our conversational exchanges are based on underlying cognitive mechanisms, whose structure and order enables the establishment of intersubjectivity between interlocutors.

In this way, the structuring mechanisms of everyday conversation can be understood as mental schemata that represent:

  • Aspects of the social life of the interlocutors.
  • Mental instruments for structuring and organizing such social information.
  • Cognitive tools for interpreting, processing and generating data during the development of communicative events.
Thought, language, conversation

Human-machine interaction protocols and tacit knowledge processes

The current expansion of VUIs as a medium to interact with digital devices reverts to a radical paradigm shift in the human-machine interaction model that has been maintained to date. For the first time since the invention of computers, machines have to learn the natural language of humans.

For the first time since the invention of computers, machines have to learn the natural language of humans.

This means that, if the ultimate goal of the VUI designer is to create an effective human-machine verbal interaction, the interface must reflect the thought patterns and socio-cultural practices embodied in the interaction protocols used by humans in everyday conversation.

To illustrate this point, we will once again use the example of voice commands, the interaction protocol applied, par excellence, in human-machine interactions since the invention of computers.

Thus, as a result of the maintenance of this legacy (which, using a term subtracted from UX design, could be understood as a kind of interactive skeuomorphism), designers most commonly approach the dialog as an exchange of commands and responses between the human and the machine, in the style of the one presented below.

User: Alexa, what are my appointments today? 
Alexa: You have three dates today. At half past nine, you have the meeting with Carlos. At half past twelve, you have lunch with Ana and at half past five you have the hairdresser's appointment.

On the one hand, although this type of interaction may be pragmatically appropriate in the current context of human-machine interaction -due to the legacy it carries and since we have simply become accustomed to it-, on the other hand, the application of this interaction protocol normally results in an unsatisfactory user experience.

The reason for this is that voice commands do not reflect, in general, the most common interactive practices used in everyday conversation and, due to their pragmatic nature, they allow a very limited expression of social and emotional dimensions entailed in human conversation.

What next?

Requests in human-to-human conversations

How can we improve, then, the user experience when applying interaction protocols that involve a more natural processing of voice requests?

One solution lies in the integration of patterns of thought and socio-cultural practices, which we commonly use in a standardized way in everyday human conversations, into the interaction protocols of the machine. As already mentioned in an earlier post, this requires the VUI designer to do some linguistic and ethnographic research before he/she starts writing the dialogue.

Therefore, retaking the concrete example of voice commands, the speech act represented in them is the request. The directive nature of requests, in which an interlocutor demands an object or service from another one, affects the possibility that the person receiving the request may feel his social face treated.

As a consequence of this, in daily conversation, requests:

Machine, I don’t understand you

  • Are most frequently articulated in an indirect manner.
  • Like any other indirect speech act, they are defined around very complex sequencing patterns.
  • In adjacent pairs of request + acceptance/denial, the second (preferred) socio-pragmatically appropriate part is the acceptance of the request.
  • They are preceded by other interactive strategies that facilitate the conditions for the subsequent fulfillment of the request.
  • The interlocutors employ negative politeness strategies to mitigate both requests and negative responses.
Machine, I don’t understand you.

An example of indirect request in human-to-human conversation

We can study, then, how some of the above described communicative premises work in a request extracted from a real conversation (originally produced in Spanish). In the first column of the example, we show the utterance and, in the second one, its pragmatic illocutionary function.

As can be observed, in order for the interlocutor to be able to comply with the politeness requirement when making a request (i.e. drafting his demand indirectly), the basic adjacent turn which we could conceptualize, subtracted from the social context, as “ Give me the notes — Yes, take them”, expands to a conversational structure composed of five turns, in which the Speaker A, by applying an indirect request in the first turn, manages to redirect it towards the production of an offer by the Speaker B in the second turn.

Cognitively, this action would not be possible if the Speaker B did not share with the Speaker A the the human thought scheme that defines the fact that when someone asks another person about the possession of an object, most likely he/she requires it. Humans acquire this social knowledge through the process of socialization and it is intimately related to patterns for maintaining social order through politeness.

Human-machine socialization: new request protocols

Is it possible to assume that, as a consequence of the new communicative hybrid reality opened by human-machine conversational socialization, interactive practices like the one described above will extend to our daily interactions with voice-enabled intelligent assistants?

Our personal view is that it would indeed be desirable for users of virtual assistants to take part into a more advanced and natural conversational experience. It is the task of the dialog designer to insert the humanity of the conversation into the thinking of the machine.

It is the task of the dialog designer to insert the humanity of the conversation into the thinking of the machine.

The writing methodology that guides the dialog design is quite simple: once we have isolated the underlying scheme that guides the speech act in human-to-human conversation, we can create more natural exchanges like the following one:

A rather effective aspect of this dialogue-building methodology is that, by grounding the interaction model in the natural interactive schemata of human conversation, VUI designers can naturally insert into conversations linguistic elements related to politeness (mitigation devices and discourse markers such as please, diverse sorts of acknowledgement tokens, tone-framing markers, etc.).

In other words, VUI designers can implement sentiment analysis and its marking to varying degrees in dialogs, if they starts from examples of real conversations.

To summarize

As we have seen, CA approaches conversation as the cognitive tool available for humans, which maps in verbal interactions the tacit knowledge processes that gear our thought and allows us to create and participate in meaningful social interactions.

The application of human conversational protocols to human-to-machine conversations is both possible and necessary. Therefore, one of the core skills of VUI designers is connected with the knowledge about the pragmatic and structural devices that provide shape to conversation. In our daily work within multidisciplinary teams, part of our responsibility is to defend the feasibility of developing conversational user experiences where human interactive protocols are implemented.

In the new hybrid interactive reality created by human-to-machine communication, the very origin of conversational protocols in human interactive practices enables their optimal accessibility by human users.

If you think that these linguistic solutions can be useful to implement your voice product, do not hesitate to leave us a few lines.


Originally published at https://www.linkedin.com on August 1, 2018.