A weekend at Microsoft’s HoloHacks 2017
“I think your cowboy just died of dysentery.”
This past weekend, the FutureTense team (me, Kacey Weiniger, Chinmay Chinara and Aakash Shanbhag) participated in the HoloHacks 2017 hackathon, sponsored by VRLA and AfterNow and taking place at Microsoft Playa Vista. The event focused on developing augmented reality experiences for the HoloLens. If you don’t know, the HoloLens is Microsoft’s AR headset, a clear visor that tracks your surroundings and projects holographic objects onto the real world.
We decided to make a social, multiplayer experience called P.A. Panic.
The App We Built
P.A. is short for production assistant — the lowliest job on a film set, usually the person who gets assigned all of the tasks that no one else wants to do. In P.A. Panic, you are working on the set of a big Hollywood film, with an angry director shouting at you to dress the actor before the shoot.
At the beginning of the experience, holographic objects appear around the “actor” (a second player) to let you know the theme of the scene — Wild West, Royal Epic, or Swashbuckler. From a nearby box, you select which holographic props and costumes match the theme. You then drag the prop to the actor, where it pops in place on them. All throughout this, a ridiculous voiceover from your director gives you the true Hollywood experience.
After each scene, the HoloLens snaps a photo of the actor wearing the holograms which is posted to our official P.A. Panic Twitter account, which you can find here: https://twitter.com/PAPanicAR This addition made the project a lot more social, and the final product was a cross between a frantic game and a wacky photo booth. If you want to see the final product, check out the Twitter (and retweet the photo of Kacey dressed as a cowboy as many times as possible.)
Wait, isn’t that just Snapchat filters?
Good question! Kind of. We found that the experience of grabbing holograms from a box and physically dragging them to the actor was a much more immersive, fun experience than a mobile filter UI. We also really played up the storyline aspect to give meaning behind the props and costumes — something we think Snapchat filters sorely lack.
Lessons Learned
Creativity Is A Strength and Storytelling Is Key
We made a conscious choice to focus on building a fun, social experience rather than a tech-heavy project. We did this for a few reasons: one, we knew that plenty of other teams would focus on making tech breakthroughs, and two, our strength as a team lies in story development and out-of-the-box experience design.
By far, the strongest parts of our project were the over-the-top voiceovers (written by myself and performed by the inimitable Zane Morrissey) and the Twitter account where we posted goofy photos of our ‘actors.’ Our project was the only one to get audible laughs from the crowd during our presentation, and people loved checking Twitter immediately after playing P.A. Panic to see their photos.
The important thing is, these are not developments that require a heavy coding background! The ability to write narratives and create social experiences transcends specific media. To take AR to the next level, from a series of tech demos to the future computing platform it has the potential to be, it’s going to take a lot more of this content-based approach. Often it feels like the barrier to entry for VR/AR development is high, but you don’t have to make a super complex project to stand out. Just make one with a heart.
Setbacks and Room For Improvement
Despite having two world-class developers on our team, we faced a number of technical setbacks. We had a few ideas to fine-tune our facial tracking (tracking is quite a challenge from 2 meters away) that we were unable to fully implement before the deadline. We also resorted to manually posting the photo booth pictures to Twitter; hopefully in the future we can automate this process using Twitter’s Unity API.
Creatively, we knew we wanted this to be a shared multiplayer experience, but we didn’t get to create an AR experience for the second player — the “actor” — that was meaningful. This is something we will continue to explore.
That being said, we accomplished a lot in 48 hours. Shout out to Chinmay and Aakash for their hard work (they are probably still catching up on sleep right now.)
Current Trends in AR Development
Hackathons are a great way to take the pulse of the developer community and find general trends of what AR developers are interested in making. Last year, when FutureTense participated in an Ayzenberg hackathon, many of the teams focused on spatial mapping and understanding — using the HoloLens to track walls and floors, and incorporating physical objects into their projects.
This year, that technical leap seemed to be mostly solved (most teams used Vuforia to recognize images and objects.) A majority of the teams were working on developing multiplayer projects sharing holograms between multiple HoloLenses. I can’t wait to see these developments progress even further in the next few months.