Project 2: Materializing the Invisible

The goal of this project, “Materializing the Invisible,” is to choose an invisible phenomena such as wi-fi, time, etc. and make it visible, tangible, audible, or some other manifestation that people can interact with.

Start Date: February 7
End Date: March 21

In order to start thinking about invisible phenomena, we did a class exercise where we wandered our building looking for things that showed signs or traces of invisible things. Some examples are shown below.

Scratch and rubber marks on the stair walls (left); spray paint on window sill outside (middle); dented pipe cover reveals person standing on it (right)

After we started seeing these phenomena more clearly we began to brainstorm, as a class, ideas that could potentially develop into the bigger project. We did several exercises that helped us to rapidly develop ideas with each other, sort of like speed-dating, and then came together after we had each worked with every person and developed the ideas even more.

Some ideas we developed included a garment that would indicate when something was close to you or approaching you, like a car or another person, for safety reasons such as if you’re riding a bike or walking home late at night; windows that could sense temperatures inside and outside the classroom and make a judgement about whether the people/person inside would like them open; phone that could sense the emotions of the person texting you and output tactile feedback.

Deciding which projects each of us wanted to tackle and how we wanted to split up into groups.

After deciding which project ideas interested us the most, Ty and I decided to explore the concept of material feedback that reveals an invisible phenomenon. Ty was very interested in the concepts of hydration and health within the context of exercising. I was interested in the concepts of sensing space and exhibiting movement within the context of dance. Thus, we decided that our invisible phenomena would be movement patterns and hydration/productivity.
Below are some of my initial explorations of movement and trying to get a sense of how movement is initiated and how this could be visualized on top of someone moving.

A first attempt at understanding movement (dark purple indicates the least amount of movement, pink-purple indicates a moderate amount, red indicates a great amount)

Two articles that relate to human movement and motion capturing.

Article 1
Article 2

Ty and I wanted to try new ways of visualizing movement, so we bought a Kinect and synced it with Max MSP via Synapse.

It was really cool to learn how Max works and to play around with some of the different pre-made patches that could serve as a foundation for some of the movement visualization that Ty and I were trying to do.

Setting up Max and Kinect

Working with code in Max

Ty and I wanted to explore more deeply how the two of our projects, a water bottle and a bodysuit, could work together to create this system of uncovering invisible phenomena. We decided to storyboard how a person might go through a day and use them in conjunction.

Narrative for Concept Video

We soon realized that this was a very ambitious plan, as we had not found the most appropriate way to represent each of our projects. My bodysuit could be made with things like wind sensors or angle velocity potentiometers, however I both did not have strong skills in physical computing and instead focused on visualizing and presenting the idea in the most clear way possible.
To clarify our story, Ty and I reframed the narrative and asked some of the dancers from my company to help us with the project.

thanks to Allana Evans (left) Sarah Deluty (middle left) Victoria Van Benschoten (middle right)

Over spring break (March 10–19) I worked with After Effects to find new ways for expressing the color shift that occurs when movements initiate throughout the body. Though I wasn’t able to concentrate the color shifts to different limbs, I learned a lot in terms of “greenscreen” editing (using Keylight) and color phase shifting (using Colorama). My cousin, who is currently working for a motion graphics and advertising company helped me get through some of the basics of these tools so that I could tie it to the knowledge I already had with After Affects.

One of the biggest challenges of this project was synthesizing all of the work that Ty and I did and presenting it in a cohesive way. As I was gathering my material for our presentation I realized that the project didn’t really have an end point: we had done many different studies in the areas of focus and movement but there was no “final product.”

Below are several of the components of our final presentation.

Stop motion animation of bodysuit (red/pink indicates larger movements, pink/light purple indicates medium sized movements, dark purple/blueish purple indicates smaller movements.

Video of bodysuit in motion shifting colors with movement

Mapping of process from concept to visualization

System Diagram of our Two Projects

List of references:

Heddoko | Real-time biomechanics for ergonomics and sports

C74 Forums: Head rotation tracking with webcam in Jitter

MaxMSP/Jitter

Tangible Media Group

Reflecting on the Project

Working through this project with Ty taught me a lot of unexpected things. I gained a deeper understanding of Kinect and of programming with Max MSP. It’s given me the confidence of trying to explore it on my own which is something I haven’t experienced before. I also learned a lot about After Effects and green-screening. This project has also helped me to see the difficulties of collaborating on a project from a distance. I think Ty and I worked well together and put in a lot of effort collectively. If I would do something different, it would be to finish the project before our break, adapt the concept to something I could actually make physically, and practice how we talk about our work and how the two projects relate to each other.