Immersive Experiences & Messy AR
Giving old tech new life
This article documents part of a 2 week long course on Immersive Experiences facilitated by James Tichenor and Joshua Walton at the IDP 2019 at the Copenhagen Institute of Interaction Design (CIID)
We take a lot of diverse courses here at CIID, but few are as highly anticipated as Immersive Experiences. For most of us, it’s our first time playing in the sandbox of Unity, Vuforia and Spacebrew — the possibilities were endless.
Our group (Mitsu Shah, Pooja Pawwar and Gaana Srinivas) chose to investigate microinteractions in the context of virtual/augmented/mixed reality, and had one week to create a project around the subject.
What makes a good (micro)interaction?
To start, we did some desk research to find products, installations and experiences that were driven by a single interaction, and explored what made them delightful and worth revisiting.
Here’s what we found. A good microinteraction:
- Can be pleasing, compelling and/or delightful
- Directs the user’s attention
- Effectively communicates affordance to the user
- Provides (instant) feedback about a completed action
- Completes the do-feel-know (interaction) loop
Obsolete, but not forgotten
When’s the last time you used something with a bunch of buttons? No, not UI buttons, we’re talking actual physical buttons.
There’s so much to love about a cassette player. The crisp snap of closing the door, the shhh of a tape rewinding, the mechanical clunk of a real button that’s so satisfying to press and release, again and again…
Before the digital era, tech was plasticky and things sometimes needed a good whack to get working again. Tech used to be messy, and we used to love its quirks.
Augmented reality is rarely messy. 3D objects rendered in conventional reality are often unnaturally pristine surfaces, at odds with the world around them.
Obsolescence + AR = Profit??
We had the beginnings of a good microinteraction in the lovely tactile cassette player, which we dug out of a box of junk in the lab. We decided to give it new life — these controls would pilot a totally virtual video player.
Hardware — Makey-Makey, Sony Cassette-Corder & Copper Tape
The three of us were far from any sort of engineer. But by studying the mechanics of the cassette corder, we were able to get it to work in our favour in several steps:
- For proof of concept, we wired the buttons externally with copper tape to the Makey-Makey which was connected to the Macbook. This way we could assign the cassette corder buttons to perform actions on the Macbook.
2. Upon taking a closer look, the mechanisms of the cassette-corder that played and stopped the tape were accessible through the cassette door. We wired this mechanism with bits of foam and copper tape to make contact switches we could reconnect to the Makey-Makey. This meant users would be able to operate these buttons (devoid of their copper embellishments) as originally intended.
3. For the final product, all the wiring was packed neatly into the body of the cassette corder, with a handy bit to control the volume also concealed into the side. It’s like we didn’t even mess with it.
Software — Vuforia, Unity, C# and maybe some Python?
In addition to learning Unity and Spacebrew the previous week, we also used Vuforia to bring our experiences from the virtual world into ours. We used YUXI — the Mixed Reality Hardware Toolkit by Sensory.cc to kickstart our journey.
As with any project, we took baby steps and kept building.
- We used YUXI sample code to map a sphere’s Z value to move when the user touched the wire connected to the Makey-Makey, which was mapped to the Macbook’s keyboard.
- A few Youtube tutorials and much confusion around digital materials and shaders later, the video was then playable within Unity. It’s a Jonathan Pillows video by Adult Swim. It was chosen to be as odds with the cassette-corder as possible.
3. Finally, we assembled the rest of the visual experience — the player’s frame and the floating objects, and focused on making it responsive to the user’s actions on the physical buttons. An interesting point of discussion was around where the fiducial ought to go.
Fiducial [ fi-doo-shuh l] n. — A fiducial marker or fiducial is an object placed in the field of view of an imaging system which appears in the image produced, for use as a point of reference or a measure.
Part of our challenge was to naturally incorporate the fiducial into our experience. So we ended up sticking the fiducial on the cassette-corder (where else would you put it anyway)
Watch the video!
Thanks for sticking around long enough to read this! Watch our video and you’ll regret having spent the time you did on processing words, like a peasant.
