Open in app

Sign in

Write

Sign in

HAI-GEN 2024: 5th Workshop on Human-AI Co-Creation with Generative Models

by Werner Geyer (IBM Research AI, US), Mary Lou Maher (Computing Research Association, US), Justin Weisz (IBM Research AI, US), Daniel Buschek (University of Bayreuth, Germany), and Lydia B. Chilton (Columbia University, US)

Werner Geyer
Human-Centered AI
7 min readMay 24, 2024

--

Editor’s Note: In the spirit of our workshop topic, human-AI co-creation, this article was written with the assistance of a large language model.

Depiction of human-AI co-creativity. Image credit: Dreamstudio.ai

Generative AI technologies have introduced a paradigm shift in machine learning and are rapidly being adopted by consumers and enterprises alike. Consumer applications like ChatGPT, DALL-E, and Midjourney are making access to these technologies widely available and are fueling the rapid commercialization of generative AI. At the same time, generative models have enabled a radically new way for people to interact with computing technology by allowing them to specify what they want rather than how it should be produced. Users are able to create specifications of the kinds of outputs they desire in natural language, through sketches and gestures, novel UI controls or other means. In a recent article, Nielsen (2023) called this form of interaction intent-based outcome specification. He argues that it is the first new interaction paradigm in 60 years, fundamentally different from batch processing and command-based interaction. Intent-based outcome specification shifts control over how computation is performed away from the user and toward generative AI models. This shift in control enables new forms of co-creativity and co-creation, but it also creates new challenges for how to best support users in creating outcome specifications that yield the desired results.

The 5th Workshop on Human-AI Co-Creation with Generative Models, held at IUI 2024, brought together over 60 researchers and practitioners from the domains of HCI & AI to establish a joint community, deepen our understanding of the human-AI co-creative process, and explore the opportunities and challenges of creating meaningful, effective, and safe user experiences for intent-based generative systems.

The central questions our workshop addressed were:

  • How do we design, implement, and evaluate intent-based co-creative experiences that enhance human creativity in their work, play, and education across a range of media types (text, images, audio, code, and video)?
  • How will user needs for creativity-support drive the development of generative AI algorithms?
  • How can the capabilities of generative models be leveraged for positive and effective co-creative user experiences?

Workshop Summary

Our workshop featured 1 keynote, 13 papers, 2 demos, and 3 posters spanning multiple thematic areas. This work was presented in two paper and poster sessions. Authors proposed novel tools and approaches based on LLMs for group ideation and brainstorming, the use of generative AI in visual arts, poetry, and music, novel conceptual models for co-creativity and evaluation of co-creativity, conversational and agent-based approaches to video editing, the application of generative AI for learning and co-learning, novel ways of interacting and discovering with generative AI, ethical considerations, and risks and harms of generative co-creative AI.

Keynote: Urbana Verba — Generative Urban Cinema

We kicked off the workshop with our keynote speaker Mauro Martino, an artist, designer, and researcher renowned for pioneering work at the intersection of art, data, and artificial intelligence. Mauro’s keynote explored the evolution from data visualization to generative art, emphasizing their distinct philosophies and aesthetics. He discussed the data age, where data was seen as a valuable resource for understanding and decision-making, citing his project “150 Years of Nature” as a prime example of using data visualization to evoke emotions and connect people to information. Mauro then delved into generative art, highlighting its potential to create unique artifacts and aesthetics using unstructured and synthetic data. He discussed the challenges and ethical implications of using AI in art, emphasizing the importance of responsible data collection and crediting original creators. He also explored the concept of co-creation with AI models, emphasizing the need for artists to contribute their own aesthetic sense and time to create meaningful art. He showcased multiple of his own generative art projects including “Strolling Cities,” demonstrating the innovative possibilities of AI in creating unique and immersive experiences. He emphasized the importance of pushing boundaries, exploring new aesthetics, and using AI to enhance our understanding of the world around us.

Paper Session 1: Co-Creative Applications & Studies

Papers presented in the first session focused on new types of co-creative applications and user studies of them.

Paper Session 2: Co-Creative Frameworks & Artistic Expression

Papers presented in the second session discussed theoretical frameworks of human-AI co-creativity and the use of generative AI for artistic expression.

Posters & Demos

Posters and demos were presented throughout the workshop, with two focused sessions at the end of each paper session. Some papers were also presented as demos.

Workshop Discussion

Our workshop ended with a discussion led by Mary Lou Maher which focused on the interplay between human creativity and generative models. One key takeaway was the importance of understanding the limitations of these models and whether they are capable of producing genuinely creative content. Large language models, for instance, may produce uncreative outputs due to their fundamental architecture in predicting the next word in a sequence. The discussion highlighted the need for theoretical constructs that advance human-centered AI research beyond just building and evaluating systems.

Examples discussed included the inadequacy of current AI models in capturing the nuances of human interaction, such as social cues in co-creative processes. One participant noted a generative model’s failure to generate diverse poetic forms, reflecting a gap in its training data that is predominantly composed of mainstream poetry. The conversation also touched on the historical neglect of earlier AI research traditions, suggesting a need to revisit and integrate those insights with contemporary advancements.

The discussion also identified a need to foster more interdisciplinary collaborations within human-centered AI research, such as by involving sociologists, philosophers, and domain-specific experts like poets and musicians. The importance of creating specialized datasets and refining model training processes to preserve diversity was also emphasized. Additionally, participants were encouraged to contribute to grand challenge visioning efforts and special issues on human-AI interaction, aimed at broadening the scope and impact of future research in this area.

Get Involved

Our workshop continued to explore the frontiers of human-AI co-creativity by exploring experimental user experiences, ethical dilemmas, design principles, and more. There are a number of upcoming events to get involved in this community:

References

Creativity
Generative Ai
Human Ai Collaboration
Ui Paradigm
Large Language Models

--

--

Werner Geyer
Human-Centered AI

Written by Werner Geyer

17 Followers

Senior Research Manager & Principal Research Scientist leading IBM Research's Strategy on Human-Centered AI. Views expressed on this site are my own.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams