What it is, how it can combine with conversation design, and where it will go next.

Multimodal design is the art and science of creating user interfaces that combine different modalities across multiple touch points. For example, you could use your voice as an input mechanism and see the output on a graphical user interface, such as a tablet. Instead of relying on just one modality, multimodal design includes all types of input and output.

Input and output

Let’s start with the basics — input and output modalities.

Think of input modality as any method of interacting with a system. This could be through your mouse, keyboard or touchpad. Other options include speech, gestures and even physiological events such as an increase in your heart rate. …