The Things That No One Tells You About Learning to Design for VR
Working at the fringe of your skill set can be intimidating. Here’s some lessons I’ve picked up along the way that might help you get started.
I was lucky enough to attend Google I/O in 2015 and took home a free Cardboard device. After spending some time flying around in space from the privacy of my living room I was hooked on the idea of VR, but as a 2D designer I wasn’t sure how I could get involved. I definitely never saw myself getting into game design or 3D modeling before this came along. It all seemed way too intimidating — like climbing a mountain without shoes, or something equally as ridiculous. Despite my initial hesitance, I decided to move forward and think about how I could approach designing for VR.
Unity 3D seemed to be the most popular design engine, so I downloaded the free version and took a quick tour. Right off the bat I was greeted with that overwhelming interface—half a dozen panels and windows with tons of options. It was so chocked full of features that there was no way I would intuitively know what to do here. I suddenly understood why people abandon the idea completely. I didn’t panic. I knew I needed help, and that’s ok.
I decided to take a step back so I could find a way to dig in and start learning. As I write this, I’m in the midst of this. It’s been a long, ongoing process. After spending a couple of months researching VR design and development and building a small demo app, here are my main takeaways about getting started.
#1: Defining your goals early can be critical for setting a learning plan.
I feel like the main barrier to entering the VR creator arena is the feeling of “where do I start?” When people ask me, I usually suggest that there are a couple of paths they could take.
First off, there’s tons of tutorials (on YouTube or online classes) that will teach you how to build a game. You might find something like “How to build the ultimate pinball VR app.” A series of videos will take you through the steps and you’ll use specific skills needed for building that game, along with learning some basic Unity skills. And by the end if all goes well, you’ll have a functional demo. This is a good method if you’re curious about VR design. But if you have a specific project in mind, you probably don’t want to spend your time making something that you’ll never use, like a rolling ball game.
If you have something specific in mind that you want to build, you could jump in with your guns-a-blazing. If this is your approach, I commend you. But if you’re skittish, like I was, you might want to take some time to watch and work through a series of tutorials that present a basic knowledge of how to use Unity, like how to navigate the engine’s interface and work with its UI and 3D elements. Once you’ve done this, you’ll have the ability to set up a basic interface, start adding in components, and while you’re building, you can tackle each question that comes up as needed. In the end, you will have built a product of your own.
With whichever approach you decide to take, ask yourself the critical questions from the get-go. Are you just curious? Do you want to get a feel for the process to see if you want to pursue it? Do you have an idea for a game and want to start building right away? Do you want to learn Unity and game dev in its totality? Knowing your level of commitment and what you’re going to do with VR is key. I’d recommend starting small. Keep your initial scope simple and you’ll avoid feeling overwhelmed.
#2: Be prepared to sink your teeth in. Learning can take some time.
Now that you’ve devised a plan of attack, gather your patience—this won’t be a quick process. Learning how to use a game development platform isn’t an easy task, especially if you’re starting with very few of the basics. Reading about VR principles and history will feel like a boat load of information too.
There could be a lot of stones to climb to get to your final destination and you’ll probably have to carve out some serious time in your schedule. Watching tutorials or tinkering with 3D art could take days or weeks. If you feel overwhelmed, don’t let it get you down. Parcel out your efforts and time. It’ll be worth it in the end.
#3: Sharpen your spears, you’re a hunter-gatherer now.
There’s a lot of info out there, but you’ll have to search for it. When I needed help I turned to YouTube, Google, and Unity’s forums. There is one big problem — what exactly do I search for? VR design and development is a huge realm. And to make it even more confusing, rules are sparse and there are a lot of ways to do the same thing.
Knowing what to look for can be more than half the battle. Video tutorials are amazing resources, and when you have specific questions you might be able to find answers on forums. Sometimes you’ll find answers, other times, more questions. Because, let’s face it, you’re still not entirely clear what your question is.
#4: Put on your mad scientist hat and start concocting.
Once you start narrowing down your problem, you’ll often find that there’s no single source that will fully answer your question. How frustrating. Part of this lies with the fact that there are limited VR standards or best practices to follow. You might find a solution that kind of works, but it might not be exactly what you’re looking for. You’ll also find plenty of outdated information, like discussions and tutorials referencing old versions of Unity with deprecated functionality. You can still learn from them, but use them with a grain of salt.
Finding the right solution might mean gathering information from a few different sources, picking out the parts your need, adapting it, and combining it to make it your own. The process requires some analytical thinking, serious patience, and a little creative wizardry.
#5: Beware of rabbit holes. Know when to stay focused.
It’s really easy to get lost, so staying focused is key. When I first started out I let myself go down the wrong path by watching videos about how to make terrain and add blades of grass to it. I backed out of that after a day or two when I realized, I don’t need this for what I want to build. I won’t be making characters move on a plane. I was planning on keeping the camera static and letting the user look around in a 360˙ view. I had to refocus my efforts.
Defining your problem is really important. And staying focused can be tricky. Usually one small question prompted me to dig a lot more, while sending me down a rabbit hole at the same time. Step back and figure out what question you’re really looking to do or you may go down the wrong path entirely.
#6: Don’t feel bad about déjà vu.
While searching around for answers, I often found myself coming back to things I had earlier dismissed. Then I’d leave and come back yet again a day or two later. Why does this feel really familiar? Oh yeah, I’ve visited this page “many times.” You might feel like you’re seeing the same thing over and over again before you realize how to make it fit your needs.
#7: Know when to embrace a tangent.
Most questions I asked elicited about a dozen more questions. Sometimes it felt like the questions were multiplying exponentially! Trying to tackle every lead could easily take you down the wrong path. I found some explorations to be useful and they helped me circle back to the original problem so I devoted some time to these (like playing with a few 3D modeling tools to create backgrounds.) But there were some that took me so far away from the actual problem at hand, that I had to struggle to find my way back. If you’re looking to do something specific, you might end up trying several different methods to get there. Which brings me to the next thing.
#8: Back it up.
We’ve all learned the hard way about backing up our files — this isn’t a revolutionary idea. Saving your files takes on a new meaning when working in Unity because, as brutal as it sounds, things will go wrong. But it’s ok. You’ll be ready when it happens if you have a system for backup and version control.
There’s nothing worse than adding a new script and tweaking a lot of UI elements then getting slapped with compiler errors. Deleting everything and backing out of your past two hours of work can be a headache.
To avoid this, I manually duplicated each viable project before starting to experiment with something new. Unfortunately, you’ll end up with dozens of projects saved in Dropbox or on your machine, and it becomes rather wasteful. A much smarter move is to implement a Git repo solution for version control — developers know this already. It makes perfect sense.
#9: If you’re not a developer, be prepared to handle some code.
As a designer, this part was tough for me. I can usually follow the logic when someone explains their code, but I knew writing it on my own from scratch wouldn’t be possible. Sometimes you will stumble across someone else’s code, and it almost what you want, but making it fit your own project may not be trivial. A battle of trial and error will ensue, and the dreaded “fix compiler errors” message will pop up again and again, usually accompanied by a cryptic console explanation. It basically means, “You’ve jacked up the script, you idiot. Fix it or I won’t let you do anything else.”
You’ll start to realize that everyone hits these walls. Don’t worry, you’ll find a way to solve your problem. Take a deep breath. Say it with me, “Serenity now…”
#10: You can’t take enough notes.
I took notes about every tutorial that I thought was relevant and saved all the links. That way I could easily go back later if I couldn’t remember something. As I was building my first demo app, I went further and took notes about every step in the process, including everything that didn’t work for me. I collected at least 12 pages in a Google doc called “Things that didn’t work” and about 15 pages worth of things that did work. By the time I was done, I had a fairly detailed protocol assembled. I thanked the VR gods multiple times for that decision. Some of you might have a better memory than I do, so this exact method may not be the best for everyone. But things get confusing pretty quickly. So get organized the best way you can.
#11: Last of all, enjoy it!
You’re learning something exciting! It may take a while to get to your end product, but the reward you’ll feel from working through the process will be amazing. There’s nothing better than reaching that “Eureka!” moment and jumping out of your chair in joy. It’s ok if your coworkers think you’re odd. You deserve that celebration.
The realm is so new that it can feel incredibly confusing, yet creatively limitless at the same time. It’s a whole new world out there and as a designer/developer you’ll be setting the precedent for the platform as a whole. Pack your patience, don’t pressure yourself too much, and keep innovating!
Need more inspiration?
I highly recommend these videos and articles for anyone getting started:
- An overview of AR/VR principles are covered by Mike Alger in “VR Interface Design Pre-Visualisation Methods” and his “VR Interface Design Manifesto”. These are two of the first videos I ever watched about VR and design — by the end I was hooked. Watch this to get yourself motivated.
- Fundamentals of the Unity GUI are covered via making a menu system in this tutorial series Modern UI Development in Unity 4.6 (it’s a slightly older version of Unity, but many of the basics still apply.)
- Background history from a pioneer in the field, Carolina Cruz-Neira. She has been working in the field of AR/VR for 20 years. She developed the CAVE virtual reality environment in 1995 and currently heads the Electronic Visualization Laboratory at University of Arkansas at Little Rock. She recently gave an excellent talk at the SDC 2016 conference called “Enhancing Technical Apps with Immersive Augmented Reality”.
- Hands or controllers for VR — which will win? David Holz, Co-Founder & CTO at Leap Motion, talks about their hand tracking technology what what the future of VR holds in “Exploring the Future of VR and Input.” Timoni West, Principle Designer at Unity, makes a case for controllers by showing off her work designing the Unity VR Editor Tool that will let developers create games while inside VR, and in her post “UX Pointers for VR Design”.
- Johnny Lee from Google talks about their tablet and mobile (non-HMD) Augmented Reality technology in his Google I/O 2016 talk, “What’s New with Project Tango”. (He takes selfies with dinosaurs too. It’s worth watching just to see him make a giant Tyrannosaurus rex run through the auditorium.) And, in an oldie but goodie from 2007, watch him hack his Wii remote controller into a VR HMD in “Head Tracking for Desktop VR Displays using the WiiRemote”.
