Towards Understanding the Design of Playful Gustosonic Experiences with Ice Cream

Yan Wang, Zhuying Li, Robert S Jarvis, Angelina Russo, Rohit Ashok Khot, Florian Floyd Mueller,
RMIT University

“Gustosonic” refers to links between the act of eating and listening within a combined multisensory experience. Sound plays an essential role in our perception of food during eating. However, sound has been described as a forgotten sense in the eating experience. This study explores the opportunity of designing playful gustosonic experiences through eating ice cream to facilitate a playful relationship with food.

Study

We present a capacitive-sensing ice cream cone called iScream!. It generates playful sounds based on the user’s licking actions. The user is free to perform any eating actions. To understand the user experience of iScream!, we conducted a study with 32 participants. We set up the cone to play four different sounds (a roaring, crunchy giggling, and burping sound to explore fantasy facilitation, food congruency, anthropomorphism and bodily response) when eating ice cream. We asked the participants to eat the ice cream as they usually do. After the player experiences the system with four sounds, they shared their experiences of iScream! with us. Based on this, two themes derived from six findings each, which detail how players explored the different auditory interaction possibilities with their eating actions while the sounds in turn modified those eating actions.

Results

Overall, we present four design tactics for designers aiming to create playful gustosonic experiences.

• Support the exploration of sound through incongruent eating actions.
  iScream! engaged with incongruency through interactive sound while eating ice cream. For example, the crunchy sound that is a mismatch with the information from the other senses when eating ice cream. We recommend designers to support the exploration through incongruent eating actions as a way to facilitate playful eating experiences.
• Support the exploration of sound by utilizing the inherent features of food.
  The inherent features of food (i.e., colour, scent, texture and taste) could be used as a design material to support playful experience. We used the inherent conductive feature of ice cream and its ongoing transformation as a result of temperature changes to support the exploration of sound through eating actions. As the ice cream was melting over time, players had to engage with the ice cream consistently, it made a sound and hence sounded different. We recommend to designers to support the exploration of sound by utilizing the inherent features of food.
• Support self-expressive eating actions through varied sound parameters.
  Sound parameters are already acknowledged within the practice of composing music, such as pitch, timbre, and intensity. We randomized sound parameters as a way to generate playful sounds while eating. Players were able to express themselves to support playfulness as part of eating experiences.
• Support mindfulness towards eating actions through localised sound.
  The exploration of sounds appeared to shift players’ attention to their eating actions. Since the sound was changing, iScream! offered players a shifting of attention towards the food, making them aware of their eating behaviours without any additional stimulus. With this in mind, we can see instances emerging of using sound-based play to facilitate mindfulness towards eating actions.

Benefits

We believe that this work’s findings can benefit various fields including healthy eating practice, design, human food interaction, and HCI. Designers and HCI researchers could be inspired to consider gustosonic experiences, especially for facilitating playful eating experiences.

For more details, please read our paper and see our presentation at CHI PLAY 2019.

Contact author: Yan Wang

CHI PLAY session:
Gustatory and Other Sensations
Thursday, 24 October 2019, 11:00–12:30

Please feel free to also write comments or questions for the authors in the space below!