Visualising complexity: AI-assisted storyboarding for user journey mapping in user research

In this article, I reflect on why I chose to use storyboarding in user research (UR), how I incorporated AI into the process, and what insights I gained along the way.

I then present the 8-step process to leverage AI in storyboarding. When applied, this process can ensure that you create detailed, engaging, and meaningful visual narratives that resonate with the audience.

So what led to the start of this journey? It started with uncovering of a complicated and multi-layered user journey during a research round. I noted that the traditional methods of capturing user journey like linear narratives or tabulated formats – which were my initial go-to, struggled to fully articulate the user’s context including the interplay of interactions between different actors/characters, the reality of busy and chaotic environment and the disruptions that may occur and hinder the use of the service or product.

I soon realised that due to the lack of contextual information, these approaches may fall short in making the impact clear to stakeholders and therefore, may risk in failing to drive the necessary changes and improvements in the product or service offerings.

Below is an example of my initial attempt at depicting the user journey of a ‘mentor’, presented as sequential bulleted steps on a PowerPoint slide, each step is complemented with quotes from user interviews.

A mentor is an experienced teacher, appointed by the school to support a newly qualified teacher, aligning with the statutory guidelines set by the government.

Figure 1: User journey depicted textually using bulleted description of steps.

Soon, the need for a more visually engaging, and expressive method became evident. I started to look for a method that was capable of capturing and articulating nuances inherent to this user journey. These included the points of interaction with the service, the frequency of these interactions, the challenges and the pain points encountered, and the offline steps in the process that were not captured in our analytics data.

It was at this point that I began exploring the idea of storyboarding, primarily due to the visual element.

I searched for examples and noted that a colleague from work, Laura Avram at Farsight, tried crafting storyboards using AI. Witnessing the potential in the approach, I decided to give it a try.

AI-assisted storyboarding

Storyboarding, a technique traditionally linked with film and animation, involves the use of illustrations or images displayed in sequence. Its primary purpose is to visualise a series of steps or events, particularly in the context of a user interacting with a specific service or product.

To create storyboard visuals, I used Chat GPT 4 (with DALL-E). I set off by employing it to help me refine the portrayal of the main characters and then depict the user journey as a story that had:

• a beginning where our user set off to accomplish a task
• a middle where our user tries different ways of getting that task completed
• an ending where our user either completed the task or struggles to do so and left the task unfinished

In addition, I sought AI’s assistance to refine the narrative for each step in the journey, outlining the action that each character was carrying out and the challenge that they may face.

Below is an example of a user journey as a storyboard, complete with images and narrative, revolving around the interactions and actions of the main characters.

Figure 2: User journey depicted as a storyboard.

What follows next is my account on the advantages of this approach and key reflections when working with AI to generate storyboards.

Navigating the nuances of AI-assisted storyboarding: advantages and key reflections

There are several advantages of using AI to generate storyboards including:

1. Rich visualisation: the move from low-fidelity to high-fidelity storyboards provides more immersive and detailed visual storytelling.
2. Efficiency and flexibility: the ability to rapidly iterate complex user journeys yourself than having to depend on a designer to draw these, saves considerable time.
3. Creativity and exploration: the ability to experiment with different scenarios and visual styles without the constraints of manual drawing or fixed imagery provides opportunity to be creative.
4. Engagement and clarity: the visual appeal and detailed representation helped in communicating the subtleties of the user’s journey, making it easier for stakeholders to grasp complex processes.
5. Empathy building by being use-centric: visual depictions of users and their context helps to better understand their experience as one can see how context may hinder or promote, and what other distractions may be intervening etc. This can be achieved in two ways by:

5.1 Identifying pain points: high fidelity storyboards can highlight user pain points more effectively than textual descriptions alone, guiding UR professionals to focus on areas needing improvement.

5.2 Visualising service touchpoints and user flow: illustrating service touchpoints helps in understanding how users navigate through a system, revealing potential areas for improving user experience.

Key reflections

While there are clearly advantages of using this method, there are also some key reflections to consider when engaging with AI to generate visual artefacts such as storyboards:

1. You are the driver: the journey of AI-assisted storyboarding isn’t about passively receiving outputs from an AI; it’s about actively guiding and shaping it. The quality and relevance of the AI’s responses were directly tied to the clarity and specificity of my inputs, making AI as a facilitative tool enhancing but not replacing human creativity and decision-making. Two things to note here are:

1.1 Prompt precision for maintaining contextual and spelling accuracy: I noticed that the more detailed and explicit my prompts were, the closer the AI’s outputs aligned with my expectations. This was particularly evident in scenarios where I needed to depict complex interactions between characters or capture nuanced aspects of the user’s journey.

Figure 3: Wrong spelling and American flag represented in a British school context.

1.2 Consistency in character representation: ensuring consistent visual representation of characters throughout the storyboard was a challenge. AI-generated images sometimes varied in depicting the same character across different scenes, necessitating additional prompts to maintain continuity.

Figure 4: Newly qualified teacher portrayed as a pupil.

2. Collaborative conversation and dialogue-based engagement: interacting with AI in the storyboarding process was far from a one-way communication. It involved an ongoing dialogue where each of my inputs was met with a response that furthered the development of the storyboard. Working with AI is an evolving process where both the human actor and the AI tool learn from each interaction.

For example, when I described a scene where the mentor meets the newly qualified teacher on her first day at school, the AI’s depiction of the scene was informed by my detailed description of the setting, characters, and the mood. The back-and-forth nature of this dialogue-based engagement made the process dynamic and creatively stimulating.

Figure 5: Mentor meets a newly qualified teacher on her first day at school.

There are two ways in which you can leverage collaborative working with AI more generally:

2.1 Seeking AI’s suggestions and probing for missing elements: one aspect of this collaboration was the role of AI in filling gaps and offering new ideas. By seeking AI’s suggestions, I often found myself considering angles and elements I hadn’t initially thought of. For instance, when detailing a scene, AI’s interpretation sometimes highlighted missing elements or sparked a new idea for the narrative flow. This interactive process of probing and refining helped in creating a more comprehensive and engaging storyboard.

2.2 Challenging AI outputs for enhanced creativity: another key element of this collaboration was the practice of challenging AI’s outputs. Accepting the first response as final was rarely the case. Instead, I found that pushing back, asking for variations, or refining prompts often led to more refined and effective visual narratives. This approach, led to outcomes that were not only visually appealing but also more aligned with the intended message and context.

3. Ethical and practical considerations: while AI is powerful, it interprets prompts based on the data it has been trained on. Users should be aware of its limitations in understanding context, especially human experiences or domain-specific nuances like country flags, religious symbols etc.

Therefore, representation of people, scenarios, and contexts should be handled sensitively and responsibly.

Based on the above reflections and learning from the experience , I propose the 8-step process that can be adopted when working with AI to generate visuals such as storyboards.

The 8-step process for storyboarding

1. Character definition:

• Know which data you are going to use to inform the narrative of the story board and its character. For me the data I used was from user testing interviews with the users of the service.
• Start by detailing each character involved in the storyboard. Include descriptions of their appearance, personality, age, and gender. Be as specific as possible — details like clothing style, demeanour, and physical characteristics can greatly influence the visual output.

2. Setting and context:

• Clearly describe the environment where the scene takes place. Is it indoors or outdoors? What specific setting is it in, such as a classroom, office, or an outdoor area?
• Include any relevant objects or elements that should be present in the scene, such as a car, plane, computer etc.

3. Mood and atmosphere:

• Define the mood of each scene. Is it upbeat, tense, reflective?
• The definition of mood can influence the colour palette, lighting, and overall feel of the images.

4. Character interactions:

• Describe how characters are interacting with each other and their surroundings. Are they talking, working together, in a conflict? Be specific if there is more than one character and assign them a name if generating a storyboard as this helps with the narrative captions.
• Mention body language or facial expressions that might convey their relationship or the nature of their interaction.

5. Narrative flow — one step at a time:

• Ensure that each prompt contributes to the overall narrative. How does this scene move the story forward or add to the understanding of a character or process?
• Think about the sequence of events and how they logically progress from one to the next.

6. Exclusions and limitations:

• Clearly state what should not be included in the scene. This could be certain objects, themes, or actions that are irrelevant or inappropriate for the context.
• Understand the limitations of AI in terms of cultural and contextual nuances.

7. Use of action and emotion:

• Incorporate action verbs and emotional descriptors to bring scenes to life. For example, “The mentor warmly greets the new teacher, who looks around the room curiously.”

8. Iterative refinement:

• Be open to refining prompts based on the AI’s outputs. Sometimes, the first response might not fully capture your vision, and tweaking your prompts can lead to better results.
• Consider each AI output as a step in the collaborative process, not the final product.

Example of a prompt following the 8-step process:

“Create a storyboard scene where James, an office manager, realised that he made a mistake when placing an order. Instead of ordering 60 notebooks, he only ordered 6! He is now looking for contact information so that he can either call or email the company to rectify his order. James is sitting at his desk, looking at his computer. The room is an office which is well lit and the atmosphere is nice and warm. James is 45 years old, Asian man who is very meticulous and happy by nature.”

Figure 6: James in his office — version 1.

See below how the resulting image changes with few tweaks to the description of context and mood in my original prompt:

“Create a storyboard scene where James, our office manager, realised that he made a mistake when placing an order. Instead of ordering 60 notebooks, he only ordered 6! He is now looking for contact information so that he can either call or email the company to rectify his order. James is sitting at his desk, looking at his computer. The room is an office which is not well lit and the atmosphere is cold. James is 45 years old, Asian man who is not meticulous and is grumpy by nature.”

Figure 7: James in his office — version 2.

Conclusion

AI-assisted storyboarding is set to redefine our approach to understanding and catering to user needs, with a pronounced emphasis on user-centricity and empathy. This methodology presents substantial benefits by facilitating:

• Illumination of complex contexts: AI-assisted storyboarding offers high-fidelity visual narratives that shed light on the context, emotional undertones, and interconnected steps of a user’s interaction with a product or service. This proves exceptionally valuable in instances where direct user feedback, such as video or audio data, is unavailable, helping to unravel layers of complexity and enhance our comprehension.
• Empathetic visualisation: through AI-assisted storyboarding, we are able to more effectively place ourselves in the user’s shoes, visualising their pain points, motivations, and experiences. This approach ensures that our narratives are not only informative but also resonate on a human level, fostering a deeper connection between users and the products or services on offer.

Would you consider employing this approach in your work? Are there examples from your experience that you would like to share, or insights on using a similar approach that you’ve found beneficial? Please do comment. It would be wonderful to hear your experiences and thoughts.