A designer’s takeaways from SXSW 2022 Pt.2

Jany Zhang

Published in

Bootcamp

6 min readMar 29, 2022

This is the second part of my takeaways —Design for Emotion. Jump to a different topic from below.

Pt.1 Responsible & Inclusive Design

Pt.2 Design for Emotion and Storytelling

Pt.3 Speculative & Design Innovation (upcoming)

Pt.4 XR and Ethics in Metaverse (upcoming)

😊 Pt.2 Design for Emotions and Storytelling

With more dimensions of interactions made possible with XR (five dimensions of IxDA: words, visual representations, physical objects/space, time, behavior), experiential storytelling experiences will be a big part of content creation and requires us to understand and design for human emotions.

Notre-Dame: Bringing Heritage to New Audiences

In the “Notre-Dame: Bringing Heritage to New Audiences” case study, Ubisoft’s SVP Deborah Papiernik and World Design Director Maxime Durand shared the story of how they recreated Notre-Dame in digital space for a broader audience, after the big fire in 2019.

Photo credit: Ubisoft (left), Theartnewspaper.com (right)

Ubisoft has seen an unprecedented emotions arisen among gamers and non-gamers after the fire — people start to share photos and memories they have with the Notre-Dame, and even try to recreate it in the gaming space. While the landmark already lives in the game Assassin’s Creed Unity by Ubisoft, they see an opportunity to expand it beyond just video games but to a wider audience of all ages, with more immersive experiences, to respond to the general public’s emotions and memories of the masterpiece.

The team adapted the 3d models of Notre-Dame into VR with more than 5000 hrs of graphic work, confirming details with historians and experts, and also using creativity when data is not available, to tell an engaging story. Later, the one hour tour was expanded to CAVE experiences so that the audience can have a bird’s eye view of the facade of the building, and now a location based VR escape room game in 2022. To create the escape game in VR where the players play the role of firefighters to save Notre-Dame, the team were constantly testing out their prototypes with firefighters that were first responders from the real fire. They tried to simulate the vocabularies and body languages those firefighters use, and restore the fire scene with smoke, water, etc to the greatest extent, making the experience more “real” for the audience as if they were there.

Origin Story: The making of a hybrid experience

Photo credit: https://lostorigin.co.uk/#trailer

Another great example for engaging storytelling can be seen from the Lost Origin Story experience. It is a hybrid experience that incorporates live performance, theatre experience with mixed reality. Originally, the team at Factory 42 was commissioned to create two separated experience: one with robot for the Science Museum, and one for Dinosaurs for the Natural History Museum. But as Covid hit, they had to reimagine the experience, e.g from museum experience into mobile AR to a warehouse, from two separate experience into a cohesive one.

The process

The space

Due to COVID, the team can’t use museums as the location as planned, so they found a warehouse as the stage, and came up with multiple floor plan designs by imagining different journeys the audience will go through, and tested and iterated. Lots of thoughts went into audience onboarding experience with Magic Leap and the offboarding experience, all of which have to be immersive and seamless to fit into the story, otherwise people will feel out of place.

2. The creation

The task of merging the robot and dinosaur experiences into one is not an easy job. The team brought learnings from early shopping mall experience, and created an engaging story / quest — the audience’s mission: break into the headquarter of Origin, a warehouse in Hoxton, and try to find the lost dinosaur bones sold in a dark marketplace. The team extracted assets from the Natural History Museum and the Science Museum, worked with experts such as dinosaur paleontologists to bring in detailed educational information with theatrical gameplay context, and worked with Magic Leap to bring the experience into XR.

3. The journey

When the journey starts, the audience will first navigate in the physical space, interact with some “fake” AI, time based laser lights, simulated body scans, and even with actors playing on the background. Once they found the dinosaur bones, that’s where the mixed reality starts — they will see dinosaurs projected into the physical space, discover new species, and finally decide what ending they want to pursue, out of as many as 25 different endings.

The three secret ingredients to make the experience happen: the audience, the narrative and technology. They audience are a major part of the story, and the narrative evokes their emotions of curiosity and magic, and technology makes the experience fully interactive and immersive in every moment of the storytelling.

The language of emotion: designing for engagement (in XR)

Understanding and leveraging the language of emotion will help content creators to produce meaningful content in XR space, like the ones above. In the panel discussion “The language of emotion: designing for engagement”, four panelists that came across design, tech, science and art shared their views on designing for emotions and ethics.

Author and educator Caitlin Krause brought the concept of “human centric VR” — VR experience that brings out emotions from people, not imposing emotions on them.

Neuroscientist Sofia Batchelor shared from a neuroscience’s perspective that emotions are provoked, not created. They’re highly tied to memories, and that’s why people learn better in XR environment. Utilizing XR for experiments can be extremely helpful to establishing the foundation for emotion research.

XR innovator Nicole Lazzaro shared the 4 Keys to creating fun in game design — easy fun, hard fun, people fun, and serious fun.

Game producer and developer Jude Dai told us how they co-created the engaging Lantern Ensemble experience with 700+ audiences. They made prototypes with calligraphy brushing experiences and invited audience to try it out while observing how they use it. Every audience was asked one question in the end of their session: “describe your emotion when you were creating the lantern”. Jude then analyzed the pattern that emerged from the word cloud, and iterated to get rid of unwanted emotions, and made intentional decisions to make the emotion meditative and not social.

To design for emotion, creators need to consider the journey the audience goes through — What’s the onboarding and offboarding experience? How might we make certain emotions last longer? (e.g Wonder) How might we design emotions that are beneficial to people, instead of manipulating or harming them? Especially with metaverse experiences, we’re closer than ever with strangers, and there are new emotions emerging with social emotions. These are still questions we need to answer, not just as creators, but also designers, developers, platforms and companies that empower creators and users.