INTERACT 2017 Workshop
Exploring gesture design
This was performed as part of the workshop ‘Designing Gestures for Interactive Systems: Towards Multicultural Perspectives’ conducted at the 16th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2017, held in Mumbai, India, in September 2017.
Background
The primary aim of the workshop was to explore “gesture design”: how can we create, learn and share a set of gestures, movements, using objects and technological tools, or maybe no technology at all. The focus was on experimentation, so we were shown a demo of a tool — como.ircam.fr that could be used to design gestures. The tools were demonstrated on iPods which when move about could generate a sound corresponding to a specific movement. Using this tool, we were instructed to come up with use cases where such a tool could be helpful.
Ideation
The first step was to explore as many ideas as possible. Using charts, we came up with multiple applications for the tool and chose the most feasible one to be our concept. Discussions with the organizers led to useful insights which we incorporated into our concept. We were given iPods to test out the tool.
Concept
Our concept was ‘Interactive Storytelling for Children’. The tool allowed us to program custom sounds for any gesture we performed with the device in our hands.
The idea was that when a teacher is narrating a story to children, the device can be programmed to generate sounds from the story. For example:
- If a scene is set near a river, the device would generate sounds of splashing water.
- To generate this sound, the teacher would perform a specific and easy to replicate gesture.
- The children would need to replicate his/her gesture.
- When a lot of children get the gesture right, the sound generated from all the devices would become loud enough to act as a strong feedback.
- In this way, the children can actively participate in the storytelling process.
Storyboarding, Prototyping and Presentation
We storyboarded the entire flow of the use case to demonstrate the working of our concept.
We then proceeded to develop the prototype. I took up the task of finding and editing appropriate sounds for our story ‘The Thirsty Crow’. The others worked on coming up with appropriate gestures for sounds in the story. We had sounds for various actions such as — the sound of a dry windy day to set the scene , the crow cawing, splash of a pebble, a drinking sound and a happy background tune for the conclusion.
We presented this prototype by acting as the teachers while the other participants replicated our gestures to generate the sounds.
