Omer Shapira presenting at Art&&Code

Art&&Code 2016 Weird Reality Notes

This past weekend I had the pleasure of attending the 4th Art&&Code conference at CMU — Weird Reality — with a focus on virtual reality.

To give you a bit of information on my angle on all of this: I was at the conference on behalf of Fake Love, where I am the Director of Technology. FL is a NY agency that was recently acquired by the New York Times, and we do high end experiential event/installation work with a design/artistic focus. FL and NYT are interested in pursuing more projects with VR and AR and MR and all that fun stuff. Personally, I’m not the biggest VR enthusiast and at times a skeptic, so I think attending a conference like this was a healthy way to hear some new perspectives and see what applications were getting people excited about in this current wave of VR.

What follows are some notes or observations from the talk that were some of my bigger takeaways. Opinions are my own. Apologies if my reading of a talk is different than yours, let me have it in the comments :)

  • Michael Naimark’s opening keynote gave a great overview of the history of related “immersive” technologies that go back decades or even centuries. We don’t work in a bubble, it’s incredibly important to understand how much our current usage is informed by past work. Also to take a look at technogies that didn’t fair so well and why. A lot of these conversations about how to use these technologies have been happening for years and we have to remember to tap into that — we don’t have to reinvent the wheel.
  • Wendy Chun delivered probably my favorite talk of the whole conference. She pulled together a lot of threads related to the problematic nature of considering VR as an empathy machine, counter to how a lot of tech evangelists have been selling the tech. Some paraphrased quotes and questions I jotted down: “Empathy shouldn’t be the goal of VR because it assumes too much about visual immersion” — “What is the political efficacy of empathy?” — “Are we witnesses or voyeurs?”. Is the lack of empathy really the problem we face in fixing social or political issues? We see images of refugees that don’t drive us to action — would somehow visually embodying their perspective change a thing? This is a must watch talk when the videos of the conference come out — too much to unpack here.
  • Golan wearing his wizard hat for all announcements. Classic.
  • Really appreciated the diversity of the conference in terms of perspectives, genders, ages, industries. A common perception from other VR (or even tech) conferences is that it is somehow difficult to make things equal, but this conference shows it can be done and done well. I hope other conferences take note.
  • Different from the empathy machine perspective, I found a lot of presented work revolved around the usage and power of VR as a source of nostalgia or memory recall. This usage resonates most powerfully for me, but I’m not sure if that’s just a personal preference. (Just last weekend, Kyle McDonald did a photoscan of a student run music venue we were both a part of from 2004–2010 — Jason Rankins, another alumni turned it into a unity experiment so other alumni could wander the cherished room again in VR.) Memorable memory pieces include: Claire Hentschker’s amazing photogrammetry experiments of wandering through abandoned malls. Sarah Rothberg’s experiments of capturing the mundane and exploring memories of her childhood home. Wendy Chun talking about how VR can be used for improved recall of traumatic events and can help in PTSD therapy for veterans. Chris Manzione’s piece “To Notice and Remember” about the difficulty in truly capturing the memory of a place. Scatter’s Exquisite Museum piece that collects work from many different locations into a single VR museum. Rachel Rossin’s VR piece that has you floating around disembodied models of her apartment and bedroom when she was a child. Char Styles and Alicia Iott’s wonderful “The Girls are Home” piece that uses a model of the artist’s childhood home as an enclosure for stereoscopic views of the inside.
  • Claire Hentschker’s work on creating a world from a child’s imagination is really inspiring — maps, creatures, sounds, motion capture-all from a collaboration with a 10yr old.
  • Salome Asega’s work with using VR as a way to look at mythologies and thinking about VR evokes the feeling of embodying a deity.
  • Laura Chen bringing humor and playfulness to the wearing of VR headsets and to the thinking of how VR could be used during daily activities.
Laura Chen’s custom google cardboard viewer.
  • Stefan Welker of the Google Daydream team presented a really informative and practical talk on the dozens of prototypes they have been working on. A great exploration of the successful UX/UI elements they have found by constantly iterating on different VR experiences. They churn out a prototype a week. This is another must-watch talk to just see the range of things they are discovering about what works — to work towards a natural language of VR interactions. Some writings are here.
  • I didn’t get a sense that there was a strong consensus from all the presenters about what virtual reality is actually defined as. This is not a bad thing. Most people tried to expand the concept far past just a screen strapped to your face.
  • Brenda Laurel is a badass and one of the more entertaining presenters I saw, which is inpressive considering the complexity of her topic and arguments. Was great to get a longer perspective on the issues we face with figuring out how to use VR for storytelling (or not). The three challenges she covered were about personal agency, dramatic experience, and capability & intent. Some of my favorite takeaway paraphrasings: “It’s not about the experience of the movie/play/VR — its about the story you tell yourself (or someone else) afterwards.” “ When you design open experiences with Freytag’s pyramid, or Freytag’s spider(with one start but multiple endings based on choice), it can be helpful to have constrained variability around choices so that the narrative structure remains. You don’t have to, but you can.” I can’t do this talk justice, definitely check out the video when it comes.
  • There were a lot of VR skeptics here. I haven’t been to any other VR focused conferences so take my perspective with a grain of salt. My experience of VR in tech media is that it is an amazing revolution — so it felt good to be surrounded by people who acknowledge it’s shortcomings and the fact that there is still a lot of work to do.
  • In the VR salon, an exhibition of 16 different pieces, Paolo Pedercini’s piece “ A Short History of the Gaze” was probably my favorite piece — the only one that I tried that made me laugh out loud. Dan Moore and Jesse Stiles audio visual environment was up there as well, great craft.
  • Someone absolutely needs to write the “How to Keep an Installation Up Forever” guide for standalone VR installations. I saw a lot of issues facing people leaving things to run on their own: menus popping up, phones dying, software crashing between runs, audio issues. Other practical or hygenic things like leaving alcohol wipes to clean between users is something that should probably be standard practice, attended or not. It’s just not a super elegant experience yet, but it can be.
  • If I had to put an average on it, I’d guess that most presenters have been working on VR stuff for less than 5 or even 3 years. Not a negative, but an important consideration when thinking about longterm perspectives on topics like these. Most of these problems we’ve been dealing with for decades, there’s some answers to this stuff out there already if we do the right research.
  • Enjoyed hearing about the process that Rachel Rossin uses to achieve her paintings that are informed by a process of 3D mesh manipulation and faulty digital scans. Also her upcoming piece for Sundance looks really really cool…
  • Lots of speakers brought up this topic in particular:
  • If I’m being honest, I think I would have preferred someone like Wendy Chun or Brenda Laurel to give the closing keynote. Vi Hart had some interesting work, but I had trouble following the thread of her talk and how it related to VR through such varied work.
  • Hadn’t seen much of Mark Skwarek’s AR work before, but there was a really wide range of interesting political applications of AR in his talk.
  • Ken Perlin’s talk had a longer tail view of the future use of VR and AR. He put forth that all human existence is virtual reality- language is the construct we have been working with the longest. Made the case that computer graphics needs to catch up to human ability to combine and edit ideas like we do with language. He also spoke about the importance of social VR — by sharing experiences and being able to talk to other humans about what you’re doing in VR, you build stronger memories and associations than if you’re just doing things solo. You go to sports games not necessarily to watch the game, but to talk about it with your friends during and after the game. “The Holodeck is other people.” Also made a point about a dystopian danger of AR/VR which is that when we can see whatever we want, it follows that we can also choose to not see things we don’t want to see.
  • Hearing about things other than the visual channel was great too. Omer Shapira talked about all of the other bodily senses that are important in our perception of reality. Jesse Stiles had a very informative talk on the process of recording and working with spatialized audio for VR and other experiences — some notes of his.
  • Having this festival connected to the VIA festival was a bonus — was great to take a night off from thinking about screens on our faces and see some large scale projected spectacles with a bunch of other humans.

Overall, I had a great time. They really packed a lot in there and gave everyone a lot to think about. Congratulations to Golan, Lauren and the crew of volunteers and others for pulling this off.

p.s. maybe there should be a conference about displays that aren’t strapped to your face someday…