Making an Interactive Music Installation
The goal of this project was to design an interactive music installation for the Guitars: the Museum in Umeå. I built an interactive installation in which guitar picks could be positioned on a table to explore their individual sounds. The distance of the pick towards the center button determined at what moment it’s sound was played.
This case study focuses on the creation process of the final prototype.
More of the earlier process can be found here.
Summary
I asked myself: “How can I develop a rather complex prototype to a level in which people can experience it?” I ended up building an installation in which I mapped a camera to a projector. I used the camera to detect the position of the guitar picks on the table in relation to the center button. Once the center button was pressed, a wave visualization was shown and the relevant sound was played.
Coding wise I translated the camera feed to the relevant output. This translation included color detection and some scaling to match it with the projector resolution.
Backstory
The final prototype was the result of a 5 week project at Umeå, Institute of Design. The goal of this project was to design an interactive music installation for the Guitars: the Museum in Umeå. The project started by visiting the museum. As a music newbie, I was amazed by the beauty of the guitars but I found it hard to understand the difference in experience when playing one or another. I noticed a clear opportunity in developing an installation in which people could experience what it is like to play different guitars.
The result of some group ideation sessions were some ideas which could be developed further. This development was not done by the person generating this idea. Instead the ideas were randomly passed on to classmates who became responsible for the final product.
The idea which I had to develop further was an interactive table in which the position and the type of guitar pick created a unique composition. The aim of the concept was to further explore the sound each guitar made.
Problems and Solutions
1. Making it experienceable
Building the concept to a level in which people could experience it was a real challenge. The main problem was that I wanted to accurately track guitar picks without sensor and actuators. I ended up having to use a camera and a projector.
By dividing the creation in small parts I managed to create separately working prototypes which I combined later. This started by creating a simple Processing sketch to create a wave effect when the spacebar was pressed. Once I succeeded, I added color tracking and further complexified my programming sketch.
By dividing the creation in small parts I managed to create separately working prototypes which I combined later.
2. Mapping the camera to the projector
Another challenge was map the camera feed to the projector. The camera had a lower resolution than the projector and it was positioned so high it picked up much more than just the table.
Matching the camera and the projector involved a lot of trial and error. Once my camera hang quite stable I looked into the resolution difference. Only after I solved the conversion mathematically, I coded the solution. My solution involved creating a variable “ratio”. I multiplied the camera values by this variable. I manually looked for the ratio value by trail and error.
//translate camera resolution to projector float ratio = 2.1; float xScaled = (projectorWidth-(camWidth*ratio))/2; float yScaled = (projectorHeight -(camHeight*ratio))/2;
3. Processing the camera feed
One last problem was that I didn’t want to directly project the camera feed but I had to do some processing before. Somehow I had to come up with a solution what the camera feed and the projector did not get mixed up.
In order to control what was being projected I coded a toggle view. First I clicked on the colours I wanted to track in the unprocessed camera feed. I stored these colour in variables. Then I toggled off the unprocessed camera feed and calculated the scaling. Finally, I projected the output in relation to this scaling.
Outcome
The final concept video can be watched below.
Reflection
Because of the explorative nature of my concept, experiencing the prototype was crucial. I aimed for presenting an experience prototype but unfortunately I did not succeed. Instead, I could only use my prototype to communicate the final concept.
I aimed at presenting an experience prototype but unfortunately I did not succeed.
One of the reasons was that the projector that I calibrated with the camera, stopped working on the night before the deadline. Besides that, the level of fidelity which I aimed for and in which I started prototyping was too high. I ended up taking too much risks and did not have a working backup prototype.
I think that any interactive installation needs to be tried out with people, before delivering it. In the best case, the installation is placed in the museum, before the final iteration. Unfortunately I did not manage to achieve this either. Instead of making an experience and trying it out, I thought of an experience before I started building. I learned that this approach is fundamentally wrong. The experience emerges during the making process.
I think that any interactive installation needs to be tried out with people, before delivering it.
Still I consider the project as a success because I learned a lot in terms of coding and project planning. Next project I will try to start prototyping on a lower fidelity. This will enable me to focus on the experience rather than the implementation. Besides that I will take fewer risks when it comes down to relying on old technology.
I blog weekly about the design process, get my articles by email here.
Do you have comments or questions on this article?
Please shoot me an email.
Martijn van den Broeck (@MartijnvdBroeck)
Martijn van den Broeck is a twenty-two year old student interaction design at Umeå, Institute of Design. He loves learning new things and to share his knowledge.
