Brief: The use of AI is expanding beyond task-oriented interactions into those that are imprecise, abstract and emotional. This project is broken into two interrelated parts that examine CUI + AI from different perspectives. For both parts 1 and 2, we are considering the poetics of interaction through form-color, type, transition, speed, etc.
Duration: 3 weeks
Part 01: Visualizing Conversational AI
Session 01: Deciding on a transcript
The first part of this project looks at how visuals can support conversations in AI counseling services. The aim for us is to develop and design a screen-based, interactive application in support of AI counsellers.
The first task was to choose a transcript that we would be basing this conversation off. Choosing these transcripts seemed like a rather arbitrary choice and in trying to rationalize this decision we started off with a set of considerations ( outcomes, agency, spectrum, modes of interaction etc.) that we would need to think of and the differentials and spectrums that fall under these.
Based on these considerations, we chose to move forward with Transcript 2. Transcript 2 represented a humane conversation to us, which spoke of a caring clinician and a grieving client. As the user describes the loss of her boyfriend, the transcript goes back and forth on the different emotions and stages of grief, building on past memories in a way to define the present and interjected with self-awareness and retrospection. Adapting this in the context of AI would be interesting — What is ‘care’ in the context of AI?
Moving forward we decided to have a sharp focus, to define this care in relation to grief and how may the AI better support a grieving individual.
Session 02: Answering the Why and What
From our discussion we defined our why as:
To stimulate conversations that mediate grief/loss so that individuals feel support, reflect, and vocalize one’s feelings.
This helped guider the rest of our decision around what our AI assistant could be key. A few key decisions we made were that our app would only focus on being a grief mediator that helps an individual through the first conversations and stages of grief, that would then connect the user to a counselor as appropriate.
Understanding our why allowed to lay three key design principles to guide our decision making process:
- Caring: Focusing on how AI can show care through verbal and non-verbal outlets; so that users feel understood, heard, respected, and loved.
- Multi-sensory: Moving beyond just voice or basic visual input or output, discovering what types of additional input or output can create a more comprehensive and rich interpersonal engagement.
- Non-verbal vocabulary: Empowering the AI assistant to understand, respond to and display meaningful non-verbal communication.
We also defined the aspects of our what and grief AI assistant/mediator could and would be able to do:
- Default visual state
- In-between states (thinking or getting attention)
- and Distance or scale
In order to make sense of this information, we found that we needed to sketch potential visual form directions. Out of our initial sketches and research on current AI feedback systems, we found the typical AI interfaces of spinning dots and wavy lines.
Session 03: Storyboarding the experience
In order to contextualize KAI and the grief experience, we storyboarded the journey of a user who would use our AI assistant in order to solidify and clarify the specific steps within the journey.
By completing this exercise we uncovered a few key points that we feel will have great impact on our end design:
- The need to start a conversation and build repoire BEFORE asking for permissions
- Ask for permission as, to add functionality to the UI
- Allow the AI to recognize private vs public space and respond accordingly
- Solidifying the fact the AI is a mediator between a counselor and individual.
- We also recognized (through research) the need to incorporate color and the impact it could have on one’s response to the device. So, we explored some common and researched color palettes.
Session 05: Visualizing our AI (KAI assistant)
We felt like these visualizations feel short especially in light of grief and our design principles. Below, We’ve included a few key inspirations.
What we found was that we wanted to explore what the AI assistant would look like super abstracted 2D form vs 3D and get a response from others about their feedback and response to the forms.
To do so, we focused on using a circle or sphere as our primary form object for our grief UI due to the metaphors around grief that discuss circles, motion, and movement, as well as, the ability to make the interaction more poetic.
Out of these initial decisions, our initial representations are below.
Our exploration into an AI assistant through an abstract 2D form
Our exploration into an AI assistant through abstract 3D from.
Session 06: Critique, feedback, and next steps
For our class critique, we decided to present he above forms without color in order to get a more clear reaction to form, feeling, and derived meaning.
Critique + feedback
The key critiques and take away that we had were:
- How can we instill the emotion of caring more
- The abstract 2D shape is more familiar and affords one’s ability to assign personal meaning to the shape and movement.
- 3D form is most interesting but because one can’t see what’s inside the sphere it feels scary or mysterious/alien-esque
- The smooth movements are working well
From the critique, we set up our next steps to develop criteria to evaluate motion and form for our AI assistant. Which are:
- Organic form
- Bubble/transparent but reflective
- Demonstrates caring
- Add warmth
- Focus on displaying the range of emotions.
Part 02: Iterations
Moving forward, with our next stage of iterations we consciously tried to incorporate the different ways of representing care, where one iteration learnt from gradients and the other experimented with transparency and symmetric rippling of water.
As we plugged these iterations together, we felt that water worked as a good metaphor that represented the calm and care we were looking for. We also felt that marrying the two approaches would work to our favor, where using the right colors will add additional warmth to the current white and grey representations.
What does water afford as a material and form? And, what can we learn from it to inform our decisions?
A key moment in the project was when we realized that we could leverage the motion and direction of ripples to represent the two key states for the AI — listening (ripple inwards) and speaking (ripple outwards).
We started to discuss the different variables we can play with in order to define the matrix and framework for our system. What remains constant and what changes?
As we tried to make some sense of the different variables, we soon came to realize that till we don’t test it out, we may still be making a lot of assumptions.
Part 03: Brand and personality
Script and Voice Over:
A big part of a VUI interface is the experience itself, which is as much about its visuals as it is about its voice and conversation design itself. We chose to go ahead with a sharp yet warm voice for our AI. Another considered approach was to design the script for the on-boarding session as well, as a way to show the initial conversation and the context itself.
Color and Gradients: Iterations for Motion:
Part 04: Contextual presentation
Feedback and Critique
- Simple motion conveyed care and meaning
- The form resonated and was understandable even without voice
- Provide more clear pauses
- Slight polish of conversational design
- What would the visuals look like at night
- Thinking forward, what other functionalities are available within the app
Part 02: Counselor connection
This documentation will be updated soon.