AI as an Interface for Creativity
The roles AI can take to help us be more creative humans.
The role of AI in relation to creativity over the last couple years has largely focused on how we can get AI to “do creativity” for us.
“Code a python script that solves X problem.”
“Generate a 5 second video of a person looking over a beach.”
“Write 10 ideas business ideas that I can pursue on the side.”
“Produce an abstract, impressionist oil painting for my medium article.”
But AI is not the creative actor in these interactions. Humans are.
These kind of interactions tend to rely on the AI as a “productive” tool, taking the context and ideas provided by the user (however much) and producing artifacts like images, text, video, etc. that meet the description, as described by the human and as interpreted by the AI. Even when producing “ideas” — what the AI generates are ideas derived from data generated as it relates conceptually to the context provided by the user.
When co-creating with another person, “context” (in addition to other factors like individual perspectives, past relationships, identity, culture, etc.) is exchanged between people to find patterns and build new ideas from. When co-creating with AI, the exchange of context between the human and the AI is crucial to building new ideas, as the relationship with AI lack those other factors. The more context added to the conversation, the higher the fidelity of the creative output.
In my previous article and research paper, I introduced a method to utilize Generative AI as a way for groups of people to “co-create” with active AI participants in co-creation workshops. These agents act as a supplement to group ideation by building understanding of the context, and providing insights and ideas through a non-human, “observational” perspective. This method was built as a way for groups to easily facilitate communication between participants, exploration of context, and design of new “things” through the intermingling of ideas and concepts between both humans and AI.
In that paper I wrote the following about group ideation vs. one-on-one:
“During initial testing it became apparent that a loose conversational style of interaction for ’facilitating’ the model through [design thinking] exercises resulted in inconsistent output quality, requiring multiple back-and-forth explanations or re-framing of the prompts. This is sufficient (and sometimes ideal) for one-on-one ideation, but could be distracting in a live session.”
In this article and in future works, I’d like to explore how we might be able to use AI in 1-on-1 interactions as a tool for creative introspection, and try to understand how we might enhance the individuals creative process through less structured and more exploratory interaction designs.
To do this, I’m going to expand on some of the ideas in my previous work and build out interaction models that explore ways AI might be able to help us be more creative, by building a sense of what advantage the intermingling between human and non-human perspectives in different creative contexts provide.
To start, let’s explore 3 roles Generative AI can take on that help people work and think more creatively: The Puller, the Pusher, and The Producer.
Role 1: The Puller
When a person is creating… something — a story, or a painting, or a website — there are questions that come with the thing being created. What do I want to write about? What can I write about? Should it be a book? Or a script? What about a video game? What do I want to make?
Answering these questions is a form of creative introspection that helps us define what exactly it is we’re making. Designers do this all the time, especially when facilitating design workshops. By answering these questions (either individually, or in a group), we end up discovering the thing we want to create by throwing all the context on the table and moulding it into something new.
The problem is, sometimes we might just have a small idea of what we want to make, and that’s it. Maybe we’ve set a direction, or know the problem we’re trying to solve, but want to explore it further. To do this, we need to ask more specific questions about the thing, which, in my experience, is much more difficult than setting that initial direction.
In a creative context where a user is interacting with the AI to build out an idea, we can instruct the AI to pull information from the user, gathering an understanding of the context surrounding their creative pursuit, so they can better understand the creative intention of the user.
In the example above, we can see this happen. If the user has a high level idea, we can stimulate their creative thinking by asking context-gathering questions. Answering these questions gives the user an opportunity to think more deeply about what they’re creating. Simultaneously, by asking specific, pointed questions of the user, the AI builds a foundation of concepts and ideas to connect, and can use this information to ask even more in-depth questions, encouraging deeper exploration of personal perspective (“what kind of challenges do I want the frog to face?”) and knowledge (“what led to the frog getting lost?”).
Another example: Imagine a designer building a concept for a sustainable housing project. In this scenario, the AI can ask probing questions about their goals, target audience, and sustainability principles, helping the designer clarify their thoughts and inform their design decisions.
By gathering this context, the AI is able to pick up patterns representative of the ideas and intention of the user, enabling it to generate more relevant ideas and analysis in response, and push them onto the user, if desired.
Role 2: The Pusher
In the example above, the AI has gathered a basic amount of context about the story being written. It’s about a frog called Lonni who gets lost in the woods looking for his adventurous sister who’s been missing for a few days.
Using this context, we can see it’s now at a point where it starts to suggest elements that drive the overall story direction. It takes what it knows about storytelling in general, the elements described by the user, and outside elements the user hasn’t thought of yet, like legendary items or hidden places, and builds a suggestion for the users creative direction.
In this interaction, the AI challenges the user, pushing assumptions, observations, perspective, and new ideas - “yes, and-ing” their creative process with insights, connections, and knowledge they may have not considered.
- assumption: “Lonni’s journey seems like it’ll be filled with growth and bravery.”
- observation: “considering Lonni’s fear…”
- perspective: “what could be a compelling reason for his sister’s venture into the woods?”
- new idea: “Maybe she was seeking a legendary item or a hidden place?”
Introducing these elements to the conversation challenges the users initial ideas, exposes gaps they might not have considered, and encourages exploration of new directions. The user, with full creative agency in this interaction, now has an opportunity to either accept or reject these challenges, and can iterate on the story as they see fit. When the AI presents potential directions or ideas, it might fit the context, but it also might not, leaving it up to us whether we decide to use it in our overall idea, or parts of it to influence direction.
Outside of story-writing and text, this kind of interaction could also be applied for musicians. Consider a multi-modal generative AI trained on digital music signals (e.g. a guitar through an analog → digital controller) and composition could be provided an audio sample of an artists musical piece. Taking the composition as it is so far, a sweet bass riff for example, the AI could then generate a potential continuation of the riff, a complementary rhythm guitar, or feedback on the riff as it is, pushing the musician to consider alternative or iterative compositions and arrangements, challenging their creative boundaries.
Role 3: The Producer
Here, the user asks for suggestions from the AI directly, and in doing so, is instructing it to produce something out of the context gathered so far. Given it’s own bias, knowledge base, and training, it can generate more nuanced and informed creative artifacts for the user to accept, reject, iterate on etc. — this role is about taking disparate elements collected in the interaction and assembling them into a tangible, coherent whole.
In this role the AI uses ideas, concepts, aesthetics, knowledge, biases, etc. it’s gathered within the interaction, and produces creative artifacts for the use of the user, aiding in developing, exploring, articulating or validating ideas.
This role is useful for summarizing longer-form interactions and producing assets that help convey the ideas discussed as well. For example, instructing the AI to produce outlines (see the example below), draft concepts, notes, or other producible artifact that be derived from the interaction between the human and AI.
As discussed above, this is the most common form of Human-AI creative interaction, so it’s not difficult to think of other ways this role can be used (i.e. image/video generators, code generation, etc.) — but it’s important to understand how this role interplays with the other two, and how the gathering and synthesizing of context between the user and the AI determines the level of fidelity in these interactions. By providing context and informing our own creativity with the context provided by AI, we can then produce higher fidelity and more creative artifacts because both we, and the AI, are more informed on what it’s supposed to be.
Hiiiii — I’m Brandon. I’m a designer and human-centered AI expert building future product concepts, facilitating people to do creative and innovative work, and exploring new forms of human-AI interaction that challenge and expand our notions of creativity and collaboration.
This was the first in a series of articles I’m writing on human-AI interaction for creative contexts, informed by my previous work; Generative AI for Co-Creation and Design Thinking.
If you found this interesting, and want to read future articles on this topic, please consider subscribing, connecting, and sharing any thoughts that come to mind.
☺
