Evolving with Gen AI: A Conversation Designer’s journey
Embracing innovation and transforming conversational experiences
Here’s a conversation I had on December 2, 2022, just two days after the launch of GPT:
My intial observation, a result of a 2 minute exploration (I had to rush off to a meeting 🫣), hardly did justice to the immense capabilities it possesses.
Despite that rushed first impression, working with GPT from its onset has truly opened up a new world for me. My responsibilities as a conversation designer have significantly shifted.
Drawing from my journey, I will explore the skills and mindset shift required to embrace this new technology, highlighting how it creates a space for you to get more creative and do more of strategic work.
The GPT Awakening
What started as a simple “chit-chat” quickly led me to explore deeper into its capabilities. I was amazed by the “conversational” aspect. Simply put it was
- Contextually aware
- Capable of generating creative responses
- Able to understand varied user queries
All of which I had tried to either incoporate or navigate as a “tech constraint” while designing chatbots. This for me was a clear indication that “Conversation Design” as a field was about to undergo a major shift. It became imperative to upskill and revamp my traditional conversation design toolkit to adapt to this rapidly evolving AI landscape.
New Skills for a New Era
Given its efficiency, the world was eager to find out “How to work with GPT?”. However, I had a slightly different PoV:
“How do I make GPT work for me?”
This questioning led me to a series of experiments as I worked to develop a new set of skills crucial for designing conversations in the age of Generative AI. The first skill I decided to learn was prompt engineering.
- Prompt Engineering
We all do a bit of prompting every day and engage with LLMs (I frequently find myself asking for ideas on what to cook 🥘). There are several ways to interact with these models and get the right output — zero-shot (one example), few-shot, chain-of-thought, etc.
Prompt engineering is essentially creating a set of instructions for AI. If you know how to leverage it, you can use it to:
- Create well-crafted responses that align with your brand and persona
- Personalize your output according to the user persona you cater to
- Iterate use cases to improve the AI’s performance
These are just a few things you can do with prompting in Conversation Design. However, it’s not all black and white in the world of LLMs. In the end, it’s a model trained on global data; it will not always do exactly what you ask of it.
2. Assessing Capabilities & Limitations
While learning how to prompt, it’s critical to evaluate LLMs’ capabilities & limitations. A nuanced understanding of these systems helps discern when to leverage Gen AI, when to stick to the traditional NLP approach, or my favorite — strike a balance between the two.
Here’s what I discovered:
- It can process huge amounts of data but can also very confidently present wrong information (AI hallucination)
- AI responses can be inconsistent
- Its knowledge is limited to its training data; it might not be up-to-date with the latest trends
- Implementing and running AI systems is expensive! The cost of API calls and maintaining new models can add up quickly, especially at scale
These insights helped evolve my approach to conversation design. AI is not a one-stop solution for everything, but a powerful, resource-intensive tool that requires careful wielding and strategic implementation.
Case Study: E-commerce Chatbot
In our e-commerce chatbot, we strategically implemented AI to enhance both pre-purchase and post-purchase experiences. For subjective queries like “I need a new mobile for gaming,” we leveraged AI to engage in a conversation, extracting user preferences and offering personalized recommendations.
In contrast, for straightforward queries like “Where is my order?”, we maintained our efficient API-based system, providing quick, accurate responses. This hybrid approach reduce response times (latency being a major issue with LLM calls), significantly improving customer satisfaction.
Key Takeaway: Carefully select use cases for AI implementation. Enhance subjective interactions with AI-driven personalization, but maintain traditional methods for straightforward, data-driven queries. This balanced approach optimizes both customer experience and operational efficiency.
3. Analyzing your AI generated content
By now we understand that LLMs are not perfect.
So the impact it has had on me was to be watchful of the output it ends up generating. You have to cross reference your outputs for accuracy and carefully evaluate if it aligns with what you wish to convey.
Given the LLMs are trained on data which affirm some cumulative human bias, it is non negotiatble to test if your multi turn conversation design, leveraging AI, reiterates those biases. For example: while designing an E-commerce chatbot, I realized that when the LLM was asked to “Show sports shoes” it generated results for men’s sport shoes without cross verifying the consumer’s requirement (indicating gender bias). As a designer it is essential to implement conversational guadrails because such instances can create an irreparable feeling of distrust and disappointment with your users.
Although it’s not perfect, it can still be of huge assistance in your design process. One has to learn how to utilise it to create a perfect package of human creativity and AI.
Balancing AI with Human Creativity
Here are a few ways I have found the balance between LLMs and my creative flair:
- Brainstorming Partner: After working on the “How?” of a problem statement and possible scenarios in the user journey, I use AI to brainstorm further. I ask it to challenge my flow or suggest instances I might have overlooked in the initial draft. This approach ensures my design accounts for potential failure scenarios and broadens my perspective.
- The Human Touch: AI can help produce content in quantity, but what about quality? AI alone cannot produce content that will resonate with the users. This is where my intervention comes in since I’ve interacted with my users firsthand and familiarised myself with their unique experiences.
- Productivity Enhancement: I offload repetitive tasks to AI and use my time to focus on developing more engaging conversational flows and personalized user experiences. For example, using AI for utterance generation or creating an FAQ database. This not only saves time but often results in more comprehensive datasets, allowing me to dedicate more energy to strategic design decisions.
As I began integrating Gen AI into my products, I noticed my conversation design process evolving. It was no longer just about maintaining an efficient workflow but about reimagining my role as a conversation designer in the world of AI.
The Mindset Shift: From Linear to Dynamic Interactions
After 7+ years of honing my craft in conversation design, I found the fundamental processes I relied on — from user flow mapping to response scripting — transformed by the introduction of Gen AI.
Concepts such as enhanced intent understanding, contextual exchanges, and ambiguty handling — which were once considered futuristic are now inherent features of LLMs. This has led to a significant shift in my approach to conversation design. I no longer find myself restricted by the shortcomings of past models. Instead of creating linear flows from A to B, I now develop strategic frameworks that encompass all possible journeys from A to Z, allowing for more dynamic and adaptable conversational experiences.
While designing for traditional chatbots (which I still do), I’d map out every possible path a user might take, creating a complex web of decision trees. But with AI, I found myself embracing a more fluid, dynamic approach.
I now develop frameworks on how we should approach a use case. Here are two recent frameworks I’ve worked on:
- Strategic Buying Journey: If a person wants to buy a new product and my chatbot needs to show relevant recommendations, along with key user insights, I would extract:
- Use case (the key requirement of the product)
- Constraint (such as budget for electronics or size for clothing)
- Preferences (attributes which are somewhat flexible such as color or brand)
These attributes dynamically change as per the user, and my chatbot is able to fetch the right product for them. I make my chatbot emulate the salesperson persona which is always there for in-time help in the shop but is not overbearing in its presence. So the shop’s aisle is my carousel of product cards, and the salesperson is my chatbot.
2. Contextual Follow-Ups: In traditional chatbot design, I would define a set of follow-ups for each user journey, typically closing intents with yes/no responses. However, with Gen AI, the approach has evolved.
Now, follow-ups depend not just on the previous bot response, but on the entire user journey. The AI analyzes the whole conversation’s context and sentiment, enabling more natural, adaptive interactions. Instead of manually mapping out each step, the system dynamically builds on the dialogue, creating a fluid, forward-moving conversation.
For example, if a user asks “Where is my order?”:
- For on-time delivery: The AI offers relevant follow-ups like “Delivery executive details” or “Reschedule delivery.”
- For delayed orders with multiple inquiries: Recognizing high anxiety, the AI acknowledges the delay and transfers to a human agent.
This approach allows for more empathetic, context-appropriate responses, enhancing both user satisfaction and issue resolution efficiency.
To sum it up…
The advent of generative AI has revolutionized conversation design, transforming it from static scripting to dynamic, context-aware interactions. As designers, embracing this technology requires us to develop new skills, balance AI capabilities with human creativity, and remain vigilant about ethical considerations.
The future of conversation design is exciting and full of possibilities — let’s navigate it responsibly and innovatively, shaping the next era of meaningful user interactions.
👋 Hi, I’m Kritika. With over 7 years of experience in conversation design, I specialize in crafting dynamic interactions across voice, chat, and multimodal interfaces. As the Lead Product Designer at Flipkart, my focus right now is crafting comprehensive conversational solutions for e-commerce. In the past I have designed voice interactions for Bixby at Samsung and creating chatbots for Finance, Gaming, and Customer Experience domains at Haptik.
