Storybook: A Platform for Sharing Perspectives (Part I)
By Scott Carter, Kate Sieck, Everlyne Kimani, and Luiza Santos
Toyota’s mission is “Producing Happiness for All,” and TRI researchers work towards this goal by creating human and AI technologies that can improve the collective well-being of people and the planet.
In the Harmonious Communities department at TRI’s Human-Centered AI research division, we build and evaluate tools that help people better understand how we are all interconnected. In doing so, this allows us to experiment and discover the best ways to make more impactful connections.
One such tool is Storybook, a web application we built to help people create compelling, media-rich narratives. We are currently testing it with users to have them share difficult experiences and challenges they have faced in their own lives with others. Storybook works by guiding people through a series of instructions to help them craft a story focused on a particular topic (e.g., barriers people face in their careers) and leverages the latest advances in generative AI to inject images that bring those stories to life.
Often, conflict stems from the visceral ways that people experience differences, and the key to creating harmony is empathy. The idea of Storybook is to help people tell engaging stories that others in an organization or community can interact with and prompt them to add their stories or insights to build empathy between authors and readers. Crucially, the challenges and issues within the stories are unresolved and open-ended, allowing others to suggest and link possible resolutions derived from their own experience or with the help of generative AI to experiment with possible futures.
In this first post of a two-part blog, learn why and how TRI researchers created Storybook!
Why Stories?
People are storytellers. As Jonathan Haidt (a social psychologist and Professor at NYU) puts it, our brains are more “story processors” than “logic processors” (Haidt, 2012). We understand and process our own lives and the lives of others via narratives that we construct and communicate to others, and we do it on multiple levels, sometimes simultaneously. We tell small stories about our everyday interactions, bigger stories about larger movements in our own careers or interests, and larger stories still about the grand changes in our societies and world at large. We have a tendency to frame almost every aspect of human action as narrative: in the discussions we have with people, in the entertainment we consume, and in the way we think internally about our own lives.
We are not just skilled at storytelling; storytelling also appears to be beneficial for us. Stories foster empathy — allowing us to connect with and understand people whose experiences differ vastly from our own (Reddan et al., 2024) — and encourage prosociality (Rathje et al., 2021). Understanding our own experiences as part of a narrative arc can help us find meaning in our own lives (Rogers et al., 2023).
Why comics?
Storybook uses a graphic novel format that combines text and images, two of the most foundational methods of human communication. The earliest forms of human activity supporting asynchronous communication — Paleolithic cave paintings and later petroglyphs and pictograms — are forms of comics (broadly defined, to borrow from Understanding Comics [McCloud, 1994], as “juxtaposed pictorial and other images in deliberate sequence, intended to convey information and/or to produce an aesthetic response in the viewer”).
But we also built the system with flexibility in mind, allowing us to swap in not only the latest and best image generation systems but also systems that generate any other type of media. We are exploring extensions to other types of multimedia experiences that incorporate 3D models, video, and audio. Our vision is for Storybook to facilitate a wide range of human communication to support different narrative content and styles.
Reconfiguration is key for research
Storybook supports adaptation beyond the models it uses to generate media and can be configured from a simple text file to take a variety of different forms. It can support three types of chapter-based narratives — a complete story, advice, and requests for advice. Each story type can be configured with bespoke chapter instructions and chapter lengths. Researchers running controlled studies to understand, for example, the impact of reading or writing a narrative on prosocial behavior (i.e., actions that benefit others) can configure Storybook in a more targeted way, with features unrelated to the study removed.
Researchers can configure it with more complete support by turning on search and advice-linking features,
as well as a feature that generates printable versions of stories for offline work.
The system also supports multi-lingual UX components and story representations!
Storybook deployments can also support a wide variety of different AI prompt instructions. One set of configuration options helps set the overall style (e.g., manga, American comic, realistic portrait, etc.). Another set of options focuses on the content of images. The first step in creating any narrative in Storybook involves selecting or creating an avatar that will appear as the protagonist in the rest of the story. Users can select pre-defined protagonists or create their own. Models vary mostly in how they maintain character consistency through each of the chapters of the story.
- Text-to-media models: These models use text-based prompts to generate media. Users can leverage these models to create their own avatars with a text description, or they can select an avatar pre-configured with a model and a pre-loaded text prompt.
- Text+media-to-media models: These models ensure that generated media represents the avatar using a preset or uploaded media as a constraint.
- Fine-tuned models: Developers can also fine-tune foundation models to generate pre-configured avatars.
Once a user chooses or creates an avatar, they write chapters of their story. Storybook automatically generates a configurable number of media representing the chapter. To maintain consistency of character and style, Storybook generates each of these media using the same model as the user’s selected avatar and a text prompt derived (using generative AI) from their chapter text. If none of the media are suitable, users can generate new ones. They can also override the automatically generated text prompt to generate images more directly. Once they find a generated media that works for them, they can select it and move on to the next chapter.
Make your own Storybook!
Storybook is a constantly evolving research tool, and we are always looking for new ways to use and improve it. If you have feature ideas or want to configure Storybook for your own deployment, please contact us at storybook@tri.global.
Coming soon
We have already tested different Storybook configurations in various contexts — from online studies across the U.S. to professional mentorship conferences in Australia and Japan. Stay tuned for our next article on these deployments!
Postscript
Most examples of Storybook used in this article cover serious topics, but they don’t always have to! Here’s a fun little story told through the lens of Storybook…
References
Haidt, Jonathan. (2012). “The righteous mind: Why good people are divided by politics and religion.” New York Pantheon.
McCloud, S. (1994). Understanding Comics. William Morrow Paperbacks.
Rathje, S., Hackel, L., & Zaki, J. (2021). Attending live theatre improves empathy, changes attitudes, and leads to pro-social behavior. Journal of Experimental Social Psychology, 95, 104138.
Reddan, M. C., Garcia, S. B., Golarai, G., Eberhardt, J. L., & Zaki, J. (2024). Film intervention increases empathic understanding of formerly incarcerated people and support for criminal justice reform. Proceedings of the National Academy of Sciences, 121(44), e2322819121.
Rogers, B. A., Chicas, H., Kelly, J. M., Kubin, E., Christian, M. S., Kachanoff, F. J., … & Gray, K. (2023). Seeing your life story as a Hero’s Journey increases meaning in life. Journal of Personality and Social Psychology.
