Lab: Go Bananas with Musical Mats!

A project from Theory and Practice of Tangible User Interfaces (TUI) at the UC Berkeley School of Information

For this lab, I explored a few other materials that have capacitance sensing and created a few unique interactions with them. I didn’t follow the instructions given in lab, but have completed all explorations. :)

Tools used:
1. Arduino
2. Breadboard with Jumper wires
3. Aluminum Foil
4. Processing
5. Bananas

This time I also tried to code the entire program on my own, with little inspiration from some previous examples.

Music Mat — Capacitive Sensing Mat that helps me play music

For this part of the lab, I used multiple aluminum foils as capacitive sensing objects and used them below the mat to create music using different parts of the same mat.

For the sound output, I used Processing and a library called “Minim” to output sound based on which part of the mat is being pressed.

This is a great resource for getting started and understanding Minim

Trial 1 — Starting and Stopping a song

To get the setup to work, first I created a simple setup to make Arduino capture capacitive sensors made from Aluminium foils, using Capacitive Sensors library, and then sent that input from Arduino to Processing. And, used that output to play sounds.

Arduino Code:

Processing Code:

Video demo:

The input from Capacitive Sensors is stored as a comma separated string, and sent to Processing.

The major challenge in the first version was a lot of exception errors, and the ability to use Minim to play multiple sounds together.

Therefore, after some iteration, we can get the code to work.

Noteworthy errors in this trial

1. NullErrorException in Processing — pops up every time to remind me how bad I am at coding. But, it’s basically, my lack of understanding of how SerialEvent() works.
2. Minim Java Exception Error — Minim wasn’t loading the files, because I forgot to copy the files to the new sketch folder. (yeah, that happens too!)

Trial 2 — Playing Drums with the mat!

In this part, I fixed majority of my code to get my drums to work as needed. Here’s how it worked:

Arduino Code:
Arduino’s code didn’t change much, I was initially getting some error parsing string, so I tried different variants, but settled on sending just one comma separated string itself.

Processing Code:
For this part of the code, I had to deal with a lot of exception errors. But, I finally got the code right.

Noteworthy Exceptions:

1. Trim(): I forgot to trim the string, and thought it wasn’t quite useful, and then, as it happens, my array wasn’t getting the second value. After spending 1.5 hours fixing the error, I tried trimming the string (as shown in many examples on the web, and boom! it was working!
2. Global vs Local Variable — Another reason why my code was throwing error was because I had declared global variables outside, but unfortunately created two new local variables in SerialEvent() function. The result — draw()arawasn’t getting values that it wanted to play sound. And, while the code was running fine, processing is receiving values from Arduino, and Proessing is printing values from array, there is no sound!
3. Not using player.rewind() to reset the sound was also quite an error. It meant that the sound is playing, but once.

After solving for these, the drums worked like magic 🎉.

Going Bananas

After trying with the mat, I thought what can be an everyday object that I can use to play drums, and well I tried making a drum kit out of bananas and it worked just fine. Here’s my banana drum kit —

Credits
This one simple button driven one time music player was super useful for me to help me understand how Minim works. :)

Info C262: Theory and Practice of Tangible User Interfaces at the UC Berkeley School of Information explores the theory and practice of Tangible User Interfaces, a new approach to HCI which focuses on the physical interaction with computational media.