Back in June we held a happy hour at the UQ office for SF Design Week 2018. After a year of remodeling and slowly making our new office home, we were excited to invite our friends and community to come celebrate our new space. It also happened to be the perfect occasion for some creative expression in the form of an interactive projection piece.
Inspiration
Projects like this have their roots in all sorts of conversations around digital “makery” experiments. As designers and developers these experiments serve as creative outlets that let us try, test, and learn new things. They also challenge us to narrow in on just one final idea, which can often be the hardest part of all.
Lots of talking.
Lots of brainstorming.
Lots of links.
Lots of talking in circles.
Lots of saying the same thing in different ways.
Lots of fear of commitment. This was mostly due to the fact that we don’t do this everyday, so we wanted it to be cool and we wanted it to work.
In the end, an experiential, audio-visual experiment became our focus. It was the perfect opportunity to kick the tires on our new brand (the colors! the shapes!) and combine it with our interests in music to make something immersive and unexpected.
Brainstorming
The shape and pattern artwork we’ve been developing over the last few months was a key starting point for this project’s visuals. A matrix of shapes and cells quickly emerged, giving us a platform to show off our colors and setting the stage for all kinds of animation opportunities.
Randomizing our colors and shapes was an awesome way give it an always-fresh, never-quite-the-same feeling. Scale changes and explosions added another layer of personality to create a more dynamic and engaging experience.
Process
The physical, interactive part of the experiment took a lot of research and logistical planning. First, we had to figure out where to place our installation. Inspired by our conference room’s existing artwork, we decided on a triptych approach, which allowed us to place canvases where our paintings once hung.
Once we had our placement, we started construction. The first question was what to use as a base. While we initially thought photo frames might work, we were worried about how sturdy they’d be with all the touching and tapping we expected. Canvases, on the other hand, gave us everything we needed. They were easy to project onto and were much less fragile, which was a nice reassurance given that they’d make their debut at a happy hour.
In order to turn the canvas into a sensor, we needed to put something conductive on the back of it. Copper tape and conductive paint seemed like an obvious fit, but were quickly ruled out due to cost and timing. A bit of creative thinking and tinkering followed and we discovered that aluminum foil and spray mount could be our winning combination. Each canvas was divided into 4 separated sections and each section was soldered to an electrode on a Bare Conductive Touch Board. This allowed each section to serve as an individual sensor, giving us a total of 12 sensors.
Getting technical
At the heart of our project is the Touch Board by Bare Conductive. It’s an Arduino board with capacitive sensors that can detect when a surface is touched, allowing you to create interactive pieces. On the visual side of things, we used the creative coding-oriented JavaScript library p5.js to create the colorful graphics that would be touch-triggered. In order to connect the touch data we got from the Arduino board to our p5.js sketch, we used p5.serialport and p5.serialserver. Finally, the p5.sound library was used for handling audio files and playback.
Our piece cycled between two different states: a grid-based state with shapes and a more free-form state featuring diagonal lines. Everything was randomly generated within those two systems, from color and shape selections to X and Y positions. To keep the visuals cohesive across our triptych of canvases, we chose to forgo using three different <canvas> elements, instead opting for just one large <canvas> element which we split into three sections using two black <div> columns on top.
Audio
We knew audio would play a big role in bringing this installation to life. We wanted to know what sound it would make when all these little shapes jump from canvas to canvas. What does this rainbow waterfall of color sound like? We went looking for an ambient sound that could worm around your head as you experienced the space; a synesthetic element that could actually convey the sounds that the animation might make.
This was not a task for the faint of heart, so we reached out to Eric Dietrick, a talented musician and tinkerer, to help us create some samples. We sent him recordings of the animations and within a few days we received a ton of plug and play samples. All of them, of course, amazing.
Obstacles
Working with a new medium with some pretty DIY variables, we naturally ran into a few obstacles. The biggest challenge for us was to make sure our sensors sent accurate data to the p5 program. There were times when a single touch would trigger multiple sensors at once because the sensitivity threshold was too low. Other times, the sensors weren’t sensitive enough so we weren’t receiving any signals; generally this was because the foil wasn’t glued properly to the canvas, or the conductive copper wire came loose from the Touch Board. Trial and error was our friend here. A lot of trial and error.
Through the course of this experiment, we learned a lot and had a ton of fun exploring the world of hardware and AV. It was fun to share it with our friends and the design community of SF.
We’d like to give a shoutout to everyone who helped make this project possible, especially Jason Dietrick for the vision, Eric Dietrick for making the audio samples, Zhihao Wu for helping out with hardware, and Phil Ruppanner for believing in us.
Both Anya Tran and Lily Zhou are a Developers at Upperquad.
Upperquad is a growing team of designers, developers and producers brought together by good fortune, great clients and a shared drive to make amazing things. We’re hiring!
