Visualizing Conducting — Lena-Lisa Wüstendörfer — a WebGL experiment
Lena-Lisa Wüstendörfer is an internationally active Swiss conductor. Her beautiful performance, her crystal-clear gesture, and her enthusiasm raise the audience to a new level in each and every one of her concerts.
Online, we had to design and develop a web experience that would translate the emotion and perfection of her concerts. To achieve such a goal, we translated the experience and enhanced it to a maximum with all the latest browser technologies.
We developed a WebGL particle system whose color and intensity are determined by the movements and choreography of the conductor.
We’ve focused on one major element as primary part of the web experience: her hand gestures. For a conductor, the hand gestures, along with the visual expression and energy, is the most relevant key to a successful piece. Through the website, we wanted to share the same experience as a spectator would have in one of her concerts. Therefore, the art of her hand gesture needed to be at the core of the experience.
We shot eleven pieces of her concerts and tracked her primary hand on each and every frame. We, then, sent this set of data onto a WebGL particle system that would, according to her motion, dance around the user’s viewport. The intensity of her motion was also taken into consideration. The more the music is getting intense, the more the particle system is expanding. We use the WebAudio API to analyse the frequency of the music in real time and adapt the particle system accordingly.
The interactivity with the user’s mouse or finger holds a major importance through the experience. The concept of joining the user’s action with the conductor’s motion added a great layer of connectivity between the videos, the particles, and the user.
We kept down the interaction layer to a minimum: click and hold to reveal. This choice was made to mirror as close as possible the experience the public has during her live concerts. The music comes first, together with the experience of that music. Then, on a click, Lena-Lisa unfolds herself to the user.
The major challenge of this project was working between a 3D set of rules (the particle system) in a 2D environment (the video). The challenge of matching the exact position of a 3D coordinate into a 2D coordinate enabled us to track perfectly the tip of her stick during her performance.
This project holds almost all the latest advancement in web technologies. By mixing a multiple video player and a WebGL experience on the top of each other, we pushed the boundaries of what web technologies can achieve nowadays. Finally, the ability to make it work on both tablet and mobile devices added a great layer of complexity to the project but came as a great output, and for all screens!
Visit the website: lena-lisa-wuestendoerfer.com
Responsible at Hinderling Volkart AG:
- Concept & Direction: Michael Hinderling
- Project Manger: Sophie Murbach
- Design: Jonathan Geissberger
- Code: Adonis Bou Chakra
External:
If you also want to be part of a project like that, don’t hesitate to apply for a position with Hinderling Volkart. We would love to hear from you!
