Spatial AR with Projection Mapping

One of the things we often take for grated is our reality, physical reality. This world is an image in our mind, constructed by what we perceive through our senses of sight, hearing, taste, smell, and touch. Can we manipulate information to make our sensory receptors detect stimuli differently and thus form another reality? At the same time, in quantum mechanics, it’s proved that reality is what we choose it to be. Is our reality real, or is it an illusion?

For the first Magic Window assignment, Chian and I are making a projection mapping project to create an illusion in the physical world. We juxtapose a piece of space by projecting an image onto the door, and create a visual effect while interacting with it. Here’s how we make it.

Step 1, take a picture of the space behind the door

Step 2, apply the water ripple effect to the image of the space in After Effects, and then export the video

Step 3, project the exported video onto the closed door using Isadora

Step 4, time the water ripple visual effect and fake the interaction

In retrospect, to make the project more convincing, I think there are some potential adjustments. First, we should take a video behind the door instead of a still image to represent the space, and pay more attention to map the position of projected image and the real scene behind the door. Second, the brightness and contrast of the projector is low. We could choose a projector with better quality. Last but not least, we could achieve the real interaction with a technique like this: “https://www.hackster.io/bareconductiveteam/projection-mapping-with-a-pi-cap-f1a134".