Warming up to AI

By Alireza Karduni, Tais Mauk, Angela Kochoska, Brian Lange and Peter Winter — These parrots get sh*t started

Special thanks to Jenna Fizel, and Ben Syverson for guidance and project support.

Everyone on the call was holding up drawings of their dream treehouse. At IDEO, this type of thing isn’t totally unexpected; this is an example of a creative warmup, a short activity that gets everyone making and sharing in order to build creative energy and psychological safety amongst team members at the beginning of a meeting or working session. What was unusual this time was that an AI-powered parrot named Doodling Dave suggested we start our meeting this way.

WTF? (what’s the future?)

A lot of companies right now are scrambling to figure out how large language models, or LLMs, are going to change the nature of work. People are trying to understand how they’re going to reshape processes and culture, or enable entirely new types of products and services. These parrots are an example that AI-powered changes don’t have to be scary or serious. If people choose to intentionally design AI-tools to embrace the unique aspects of their organization’s culture, they can be used to help strengthen interpersonal relationships and make work more enjoyable overall.

Our team of data scientists and designers set out to explore this space for IDEO, but we didn’t start by focusing on warmups. We had four weeks and a brief: How might we use Generative AI to help the people who lead our design teams? We used our knowledge of common pain points, focused on the strengths of LLMs, crafted a set of 12 concepts, interviewed 5 IDEO project leads, and tried out several promising directions using rapid prototyping. In the end, we found that creating creative warmups was a pain point for (some) design teams that LLMs are uniquely good at addressing. Better yet, it’s an area that has a strong fit with IDEO’s culture.

The blessing and curse of warmups

IDEO has a longstanding culture of using creative warmups to start meetings. We’ve written about their power for shaping work culture, helping make decisions, and building creative confidence. My personal hunch for why they’ve persisted so long is because the people IDEOers collaborate with are constantly changing as we jump from project to project. These activities help us rapidly get to know and build trust with the people we’re working with. As the pandemic has caused many of us to shift to remote and hybrid work, creative warmups have only become more important.

Despite their benefits, our team learned creative warmups sometimes create an unwanted burden on our project leads, who often feel responsible for shaping team culture. “If someone could just come up with creative warm ups for me. I cannot think of another one,” an IDEO project lead said during an interview with the team. Another lead said: “When a project gets stressful, [warmups] are the first things that I stop doing”. When teams needed a warmup the most, they were jumping into things cold.

The parrots are born

Generating warmups looked like a perfect use case for an LLM. The ability of the technology to take a wide range of natural language inputs and generate many variations would help keep things fresh. And some of the most challenging parts of the technology like hallucinations turn into strengths (or at least not pitfalls) when creating warmups that encourage suspending disbelief and getting a little silly.

Obviously, we needed to build some parrots.

Ok, maybe the jump to parrots does require some explanation. A common challenge when designing interactions with LLMs is that when a system can “speak” like a human, people expect it to have human level intelligence which it doesn’t have. Using the metaphor of parrots gives users a calibrated level of expectations for how smart this tool is. It’s not a human, it’s just a pet bird. A bird can make mistakes, they get things wrong all the time.

Parrots in particular are a nod to the LLM nickname of “stochastic parrots” coined by Emily M. Bender in an influential 2021 paper highlighting the risks of LLM development and more responsible ways to develop them. Parrots have also played a role introducing new technology to the public in the past, as the animatronic parrot from Disney’s Enchanted Tiki Room was Disney Imagineers’ first application of audio animatronics back in 1963.

How to use the parrots

We created a streamlined interface that would make generating warmups easier and more effective than doing it from scratch in a blank prompt box.

The user follows 3 steps:

1. Select the parrot you want to create your warmup (each parrot specializes in different types of activities)

2. Feed the parrot any type of theme you want the prompts to riff on

3. Do one of the suggested creative warmups with your team.

These steps simplify many of the difficult parts of both building an effective LLM prompt and building an effective creative warmup:

Context

The best creative warmups reflect some context about the team and what kind of work they’re undertaking together. The different parrot specialties reflect common needs on IDEO projects. Depending on the meeting you might be more interested in fostering connections (show and tell or sharing something personal) or in getting everyone’s creative juices flowing (Storytelling, Drawing, Building).

The ability to provide a theme adds another layer; the user can use a theme related to the work, something they know about the people participating, or get a random one if they can’t think of anything. It gives more control to the user and makes the results a bit more relevant and unique.

Examples

Good prompts often also have examples to help shape the results received from an LLM. This technique is called few-shot prompting. Luckily, we already had dozens of examples from an earlier prototype of a creative warmup chatbot. We put these into an Airtable database and categorized them. Using the context the user provides, we grab a random set of the most relevant examples and include them in the prompt.

Personality

Finally, just for fun, we added in a specific description of each parrot’s personality to match the parrot’s specialty. Why wouldn’t you want a drawing activity to be run by a parrot imitating an elementary school art teacher or a fantasy author?

A beak behind the curtain

The app is a React front-end hosted on Google’s Firebase, and a backend is powered by a customizable prompt template that is sent to OpenAI’s API. The prompt templates have sections that take the user input from the front end to help flavor the wording of how the output should be written. The LLM creates a JSON of three warmups (title and description) so that it’s in a format that’s easy to display on the React front-end.

This is what the full prompt template looks like with all the pieces put together:

def getParrotPrompt(personality, tone, theme, activity, examples):
  return_character = "\n"
  system_prompt = f'''
    You are a meeting facilitator have the personality of a {personality}. "
  '''
  user_prompt = f"""
    Make five simple and silly prompts for creative warm ups on the theme {theme}.
    The warm ups should all be based on {activity}. 
    For each one, give a silly intro in your specific tone and personality!
    Your tone for the warmup is {tone}.
    Here are three examples of such warmups for inspiration (don't copy them!):
    {f",{return_character}".join([f"{i+1}. {example}" for i,example in enumerate(examples)])}
    Make your response as short as possible.

    Format your response as ONLY a JSON array like the following:
    [
      {{
        "Title": "Title of the warmup",
        "Instructions": "Instructions of the warmup"
      }},
      {{ 
        "Title": "Title of the warmup",
        "Instructions": "Instructions of the warmup"
      }}
    ]
    """
  print(user_prompt)
  return system_prompt, user_prompt

An earlier version of this prompt tried to include some extra parrot noises in the warmup descriptions. The middle of that prompt looked more like this:

user_prompt = f"""
[... skipping this bit ...] 

You're a parrot with the personality of a {personality}.
Your tone is {tone}, 
but you also include bird noises, like 'Squak!', 'Chirrup!', 'Cheep!', 'Hoot!', 'Croak!', 'Screech!' etc."
Make five simple and silly prompts for creative warm ups on the theme {theme}. 
For each one, give a silly intro in your specific tone and personality!

[...and skipping this bit too ...]
"""

Unfortunately, this led to TOO MUCH PARROT. It would ignore the chosen theme and just make parrot themed warmups every single time. That prototype led us to a realization: a skilled facilitator, parrot or otherwise, would respect the theme and not make every warmup about themselves. That’s why we cut out explicit mentions of parrots inside the prompt.

Pointers from proud parrot parents

If we had to summarize what helped us get from “nothing to parrots” in a matter of weeks for other teams looking to apply LLMs in their workplaces, we’d boil it down to this advice:

1. Look for opportunities in areas of your business that are ripe

The project began from searching for needs within IDEO’s project lead community, an audience we know is essential to delivering great outcomes for our client and often has a lot on their plates. Think about where the roles in your organization where people are stretched particularly thin.

2. Come with ideas and be ready to listen

The team generated a bunch of potential avenues to pursue and came to our first conversations with project leads with ideas to react to. By getting concrete early, sharing many different directions, and being genuinely curious about what our users needed, we avoided talking in circles and quickly got to practical things we could test.

3. Start getting functional fast

When prototyping with LLMs is just a free, no-code-required prompt box away, there’s no excuse to wait on wading into the technical feasibility of a solution. When our research participants pointed to a concept and said “I’d use this if it worked well…” we took that as a sign to stop talking and start making simple versions. Even simple prompting can help you see how well you’re doing and where the technical challenges are.

4. Weigh desirability and potential payoff against the technology’s strengths and weaknesses

While LLMs are an exciting new technology that are applicable to a surprising number of problems, they do have their limitations, including the ability to generate misleading or incorrect information and apply invisible and dangerous bias to decision making. Considering whether the problems you hear from users magnify or shrink these limitations can help you pick the right problems to work on.