Thank you all for the extremely warm welcome for my debut piece here on Medium! The work condensed in You’re Doing Mixed Reality Wrong took years of research and experimentation to solidify and express, so for this follow-up just a few months later, I want to focus less on a specific what and more on my process of how. Ahead, I’ll talk about my research process and what I learned from making AR drawing videos at my first ever artist residency. (Hint: thoughts on HoloLens cinematography!)
Ecotones
My research starts with an ecotone. In ecology an ecotone is the venn diagram overlap zone created by two or more ecosystems rubbing up against and transitioning into one another. The neighboring systems in my research are usually one from the technology world (VR, AR, Smartphones, etc) and one from the cultural world (drawing, video editing, sketchbooks, dance, etc). Instead of finding preexisting occurrences to study, I create artificial ecotones in the lab by vigorously rubbing the two subjects together then studying the aftermath. This process always yields unforeseeable outcomes which teach me more than projects based on preconceived assumptions.
So lets mix one part Drawing with one part Augmented Reality and see what bubbles up…
Starting Abstractly
I made many more AR drawings during the residency than I will show you in the 3 following short videos. This is me feeling my way through learning to draw in the Hololens and how those drawings lived in the world. I recommend you watch each video then continue reading.
In fig. 1 we see that drawing in this context can be a whole body gesture. As an experienced dancer I knew going into this project that documenting my movements while making drawings would prove insightful so as well as recording from the HoloLens I also set up a standard camera and layered the two capture methods. Fig. 2 is also a twofer, but instead of line creation, it studies body movement for video capture to explore a preexisting drawing scene (more on HoloLens cinematography later). In fig. 3 I learned to go with the initially frustrating bugs in the tools I was using instead of trying to fight them. Positioning errors, flickering, and long perfectly straight lines all appeared constantly. Despite these bugs, with time I began to relax and draw daydreams into the mundane world around me as in fig. 4 with the cutting board. That connection to the real world expanded in fig. 5 when I tried drawing in the real world with a flashlight.
Ghostly Visitors
Because I had the HoloLens on all the time I ended up drawing in lots of mundane, relaxed situations: cooking, washing dishes, playing with the dog, and, as in fig. 1 in our second video, chatting in front of the fire. For 10 years I have avoided making figurative art, sticking to what felt like the safer waters of conceptual and abstract work, but here it felt easy, simple, right. And after that first tracing I haunted the house with holographic figures. The day after a migraine took me down I drew fig 2. a portrait of myself unraveling in a chair. Then fig. 3 emerged… a holographic body I could climb inside and animate with my own. With fig. 4 the drawings became detailed portraits of invented people, transgender and non-binary people, who emerged from my hand full of thoughts and feelings even I didn’t know. I realized I’d avoided figures in my work all those years because I was afraid of revealing too much about myself thru them. That thru my drawings of people the audience, you, could see me too clearly, my missing pieces and broken parts and hollow crevices. I was right, but I’m not afraid anymore. Fig. 5 took it to bed. I filled my bed with a bright pink figure then climbed in after them. This drawing drives my mind toward questions about consent, about blow-up dolls and virtual sex and avatar bodies, toward body dysmorphia and top surgery, and how we touch each other across the digital expanse.
Drawing in AR is site specific, tissue fragile, and lives in your feet as much as your hands and its here… not trapped on the other side of a frame from you.
HoloLens Cinematography
Unforeseen learns! My favorite part!
Because the drawings themselves can’t be saved, I could only capture video documentation before they were disintegrated by low battery, lost tracking, or software crashes. That meant even though I went to the residency focused on AR and drawing I was actually making AR drawing videos and thinking about embodied cinematography.
Here’s a migraine recovering M to video explain more:
Speculative AR
Because of the fragility and limitations of the HoloLens drawings, I spent my residency drawing until the Hololens again needed charging then animating speculative AR drawings using a standard camera and Premiere. Here are the 5 that survived to completion:
Eventually I began to imagine the HoloLens and Premiere processes as one tool. With this new tool for making animated AR drawings I could fix a drawing tip to any body part, or even a fixed position relative to the headset, then let my arms swing and play while I danced around to draw the path. This would just give me a path with a thin white line and I could then move its anchor points and change the curves. Using a time scrubber in my left hand, I could move along the animated path, opening and closing my right hand to indicate the size of the brush, and using voice commands for color changes, leaving key frames as I went. With color done, I could watch my dance draw itself in the air from lots of different angles speeding up and slowing down different parts by placing in and out points on the drawing itself then simply stretching or scrunching the entire time scrubbing tool in my left hand. AR magic!
Did you notice that part in the story just where we went from making art to making tools? I make art to make tools to make more art, systematizing thinking outside the box.
Why examine process?
Considering how we make technology (or clothes or dog food or educated adults or anything really) is crucial because the processes we use change outcomes in dramatic ways. Take the end of my process for this project: figuring out whats good, which drawings get to see daylight and which get pruned. How success is tested in a field will, like in evolutionary fitness, shape its structures and types.
Think about learning something new: it’s commonly held that the systemic expectations and beliefs of the environment shape what gets learned. It doesn’t matter if we are talking about machine learning or human learning. Two examples:
- How will paying teachers more when students score higher on standardized tests bias learning outcomes?
- How will focusing bots only on high click volume bias results?
Expectations bias outcomes. This is as true in learning as it is in technology. Expectations construct extrinsic rewards in order to shift outcomes toward confirmation of a preexisting bias:
- How will hardware design which expects users to sit in front of computers influence health?
- How will leaving a public internet forum unmoderated shift social outcomes?
Perhaps millions of lines of digital ink have been spilt on the fact that, desire for, hot takes on, and reasons why the tech industry is a mess (if you seriously need me to enumerate the tech industry’s many deep rooted problems then… Hello! Welcome to the neighborhood! Tea sometime?). There are lots of ways that need changing, each a process, each with its own test. It’s crucial we have the skills to unfold processes and systems, especially our own dearest held, to examine the standardized tests and un-moderated forums and tests for success from which our actions and ideas and work dilate.
So for transparency: here’s my test. The test I use for measuring success. It’s easy. I make art to make tools to make more art to make more tools, so my test goes like this:
- Does making this art help me learn something new or clarify something out of focus?
- Does this tool improve mental, emotional and physical mobility and stability?
That’s it. I’m sure it’s flawed in a least a billion ways. All tests are. That’s why we are public about them and willing to change!
Thanks for reading!
The goal of my research practice is to make art and tools with a medium we barely have a grasp on yet. To do that, I am continually folding in older, better understood forms of making, linking progress to tradition, seeking productive friction. Kneading together knowledge from previously segregated fields makes for richer soil from which to grow our future.
Let me know what you think about these AR drawing videos, embodied cinematography ideas, and what your tests are for success!
