VR on the Horizon
At the time of writing this article, we are at one of the most exciting times in technology since the dawn of the smartphone. Virtual reality is coming in many shapes, sizes, costs, and targets. We’re looking at a broad spectrum of focus from almost all of the big players in the technology industry, and it will be interesting to see how it will pan out. In this article, I want to express my observations from across the industry, while giving a brief history of modern virtual reality.
Palmer Luckey and the DK1
Like many great technology stories, this one begins in a garage in California. Palmer Luckey, a 19-year-old kid in Long Beach, was tinkering in his garage trying to build a head mounted display for gaming. He had a passion for gaming and wanted to build an immersive head mounted display for personal use. He documented the process in a forum post titled Oculus “Rift” : An open-source HMD for Kickstarter. It’s a fascinating read and I especially like the part where he says:
I won’t make a penny of profit off this project, the goal is to pay for the costs of parts, manufacturing, shipping, and credit card/Kickstarter fees with about $10 left over for a celebratory pizza and beer.
If you know how this story ends, you’ll realize that this is a really funny quote, but we’re getting ahead of ourselves. Palmer made a really good Kickstarter video and it was a huge success- one of the most funded projects in Kickstarter history. Looking back on the original campaign video and seeing all of the compliments from across the gaming world, it’s easy to see why people were so excited.
The Oculus DK1 shipped out and inspired millions. Oculus created a site where people could share their creations, which mostly consisted of simple games and “experiences” that really showed people the power of VR. It was here that I first experienced the new frontier of virtual reality. At a friend’s barbecue late at night, someone pulled out a DK1, hooked it to an ASUS laptop with an XBox 360 controller and started up a demo of Lost in the Rift. The first development kit was crude. You had several wires connecting the device and you had to measure the distance between your eyes and input it to the software. Once we got it setup though, all of that faded away.
Playing Lost in the Rift was the single most mind-blowing experience that I have ever had. I remember that I had to walk through a field to find an entrance to a cave, and I realized that I look down at a map- which was in my virtual hands. I could look around the screen moved with me. If I heard a sound, I could turn to see it. The level of presence that I felt in that moment was amazing.
This isn’t to say this experience was not without flaws. The setup was overly complicated. The screen resolution was so low, that you can see each individual pixel (the so-called Screen Door Effect). Some people who tried it got extremely motion sick, and had to stop before they could finish the experience. Still, with all of these issues, I could see the potential and the future of this platform.
Cheap VR for the Masses
People were getting really excited about the future of virtual reality, and what Oculus will bring in 2014. However, the next big innovation came from Google. At their annual I/O conference, they announced the Google Cardboard- an apparatus that can be used to turn your smartphone into virtual reality headset. This was huge news! For something that only costs a few dollars, you can take what most people already have in their pockets and show them the next generation of computing.
The Google Cardboard was a breakout success. Companies could now build applications and distribute them through the channels that they were already using, mobile app stores. The headset was so cheap that they could be given away for free as promotional items. I grabbed my first Google Cardboard from Unofficial Cardboard and played with it for a long time. I tried a bunch of different apps on a bunch of different phones. While the experience wasn’t optimal, I came to a realization:
- This is the easiest way to VR to the public. They already have 90% of the hardware that you need in their pockets.
- Having an untethered solution is far superior to what I had seen with the Oculus. Not having to worry about cables, or the computer specs that you are hooking into is a huge deal.
- The screens on our phones aren’t anywhere close to what we need. The screen door effects is still there on every phone I had tried. The colors smeared. The screens were dirty from all of the non-VR use.
- Crappy hardware gives a crappy experience. I got several more cardboard headsets as promotional items, and I could really tell the difference. There was no universal size so certain apps caused double vision on certain phones. The lenses were terrible- most of them arrived scratched and ruined the experience.
Mobile is the future. It is what will bring this technology mainstream. Cardboard started the trend, but it’s not the final answer.
Refine, Improve, Innovate
People were excited. Innovation was everywhere. Oculus released their second developer kit, DK2, and introduced us to positional tracking. The DK2 was covered in a series of infrared LEDs which are tracked in by a camera- a system that Oculus calls Constellation. Using this positional tracking technology, the software becomes aware of where the user is. This allows for even more presence inside of virtual reality experiences. For the first time in a driving simulator, the user could lean forward to get a clearer view of the dials on the dashboard. Users could peak around the corner to see what is there.
I used the DK2 for the first time at a tech conference at Microsoft’s Redmond Campus. I saw it as a really great successor to the first Oculus experience that I loved so much the first time that I tried VR. The resolution was much better, but not good enough. The tracking was much better, but not good enough. Overall, I thought that Oculus was making progress in the right direction, but I could feel that more was coming.
Augmented Reality is the Holy Grail
I don’t want to continue this essay without mentioning a parallel effort that is being developed, Augmented Reality. This really started in 2013 when Google introduced Google Glass, a wearable set of glasses with a camera that showed contextual information for the world around you. I tried the Google Glass Explorer Edition at work one day, and while it was impressive tech, I wasn’t sure that it would catch on. The navigation is clunky, there was a trackpad that took out of the experience. It looked crazy. No one would walk around in public with this thing on their heads- although I’m sure a few generations could have it looking much better. But this biggest issue was the camera- people don’t want anonymous camera pointed at them all the time. You started to see bars and restaurants banning Google Glass. YouTube videos surfaced of Google Glass wearers being attacked. I think that The Daily Show expressed it best. It was obvious that Google Glass wasn’t the final answer.
Augmented Reality kind of disappeared for awhile. Everyone was focused on VR- until Microsoft jumped into the picture. They introduced HoloLens alongside Windows 10 and blew everyone’s minds.
This was obviously a tech demo that wasn’t ready for prime time, but it was impressive. Bloggers got to use an early version of the device, and their reactions were overwhelmingly positive.
Another company, Magic Leap, has something brewing as well. They’ve shown their prototypes to billionaire who have likened it to alien technology. No one has really seen what they are doing outside of some YouTube demos though, but whenever they do introduce something, I expect that it will make big news.
Microsoft eventually showed off its first version of HoloLens (I even got to wear it!), but it didn’t live up to the hype that they created with the first demonstration. It’s obvious that augmented reality is the holy grail here, but I think that it is going to take a few years to mature.
Palmer Sells Out
It’s worth mentioning here the single biggest moment in Oculus history so far. In March 2014, Mark Zuckerberg announced that Facebook had acquired Oculus for 2 Billion dollars. See that Palmer quote at the top of the page is funny now?
Side note- here is a really funny post on /r/oculus that pretty much explains what people thought of this announcement.
People were livid. They accused Palmer of “selling out”. They assumed that their prized open gaming platform would be used for Farmville. They thought that there would be no privacy, that Facebook would sell their personal data, that Zuckerberg would be calling the shots from now on.
One of the loudest voices was the creator of Minecraft, Notch. It was everyone’s dream that Minecraft would come to VR, but it didn’t look that way:
In retrospect, people overreacted. I think this was actually a good move for Oculus. They got a ton of publicity and cash. Overnight, they became one of the biggest names in Silicon Valley and their research budget was able to balloon in size. They made some acquisitions of their own and hired some really bright minds.
This is where the platform wars start. The line was drawn here with people who supported or hated this. Personally, I like Facebook and I think that the privacy issues are overdrawn. I think that there is a large portion of VR enthusiasts who see VR as simply a gaming platform. I think that Facebook has the ability to take VR to the masses and create social content that we have never seen before. Sure, there are plenty of complaints that could be made, but I think the good outweighed the bad here.
Use Your Hands
Now that virtual reality could track your head, people wanted to use their hands for input as well. Kickstarter projects started showing up everywhere for hand tracking- and there were a lot of different implementations.
Obviously, the best thing to do would be to track hands and fingers in space without having the user hold any type of controller. The biggest name to come out so far is Leap Motion who made a miniature sensor that could monitor your hands similar to the XBox Kinect and create a skeletal tracking mechanism. This was really interesting, and continues to develop- especially since Leap announced their new Orion SDK a few days ago. Another company called Nimble Sense created a Kickstarter, but ended up cancelling because they were acquired by Oculus.
It was around this time that we first saw the biggest competitor to Oculus, the HTC Vive. It was unveiled during Mobile World Congress in March of 2015 and seemed to the answer to everyone who was unhappy with Oculus. The Vive included “wands” that could track your hands, and had a new system for positional tracking called “Lighthouse” that is far superior to Oculus’ Constellation. Lighthouse places laser emitters in the corners of rooms. Instead a camera tracking where the user is in space, the Vive uses the lasers to calculate its own location. This is much cheaper, faster, and less demanding on the PC.
So here it was, the HTC Vive. A collaboration between a massive hardware vendor, HTC, and Valve- the most respected name in PC Gaming. Vive was coming to market at the same time as Oculus with hand tracking, with Valve, and without Facebook.
Oculus was ready to answer though- with Oculus Touch. A set of motion controller that uses Constellation to track your hands.
There is a platform war brewing, and it is getting ugly.
Oculus/Facebook had been working with Samsung on a mobile VR unit, the Gear VR. The Gear is an attachment for certain Samsung Galaxy phones that adds extra sensors, and bypasses the operating system to allow for straight-to-CPU higher-quality mobile VR. When I first tried the Gear VR, I was blown away! This thing had a better picture and response than the DK2! You could pair it with a Bluetooth gaming controller and have some really great experiences. Oculus software started to really shine when the Gear VR came out. They introduced Oculus Home- a launch for experiences such as theaters, games, arcades, movies, and any other future VR content.
This was the first and most polished VR experience that anyone had seen. There were amazing experiences- something as commonplace as watching Netflix became amazing by transporting you to a mountaintop cabin watching on a huge screen. Oculus Arcade and Oculus Cinema transported you a different world to watch movies or play old games.
There were same major drawbacks though:
- The Gear VR overheats, constantly. The mobile phone is forced to do a lot of processing. This kills the battery and will kick you out of your app for overheating every few minutes
- Oculus Home is a nice experience, but it is a closed garden. Only apps which are approved by Oculus can go here and Oculus retains a cut of all of the app sales
- Not everyone has the latest Samsung Phones. They are expensive, but for good reason. They have high-refresh-rate OLED screens, which are needed for VR. The LCD screens in most phones (like iPhones) cannot refresh fast enough to have a comfortable experience in VR.
After countless delays, Oculus and HTC were ready to show the world their consumer headsets. Each had released two development kits and had legions aligned on either side. Oculus was first- they were ready to unveil their consumer headset, the Oculus Rift.
Oculus started by unveiling their required PC specs, which were much higher than what either of the previous development kits required. This upset some people, since they would have to upgrade the hardware that they had been using in the development kits to run the Rift. However, Palmer Luckey came out and said that the cost of a PC that meets the specs along with the Rift will cost as little as $1500 USD. A couple of the tech blogs took this quote way out of context and reported that the Oculus would cost $1500. This really upset people, and when they took to Twitter, they asked Palmer why the headset was going to be $1500 when the dev kits were $300–350. This is where Palmer made the infamous “it’s in the ballpark” quote- he was saying that it was closer to the development kit prices than the $1500 that was being reported.
When the Rift bundle pre-orders went live, people were shocked. The price was $599- not in most people’s ballpark of $350. On top of that, Oculus was bundling an XBox One controller and two games that people felt were unnecessary and raised the price- such a high price when the Touch controllers are not even released put the Rift out of non-enthusiast price range.
The Oculus price turned the line in the sand that had been drawn between the Rift and Vive into a canyon. People took to social media enraged- they felt betrayed. Palmer took to Reddit to do an AMA as a form of damage control. Still, the damage was done. Prices were high, shipping prices were high, taxes were high, and expectations were set much lower that reality.
If you read through his post, Palmer apologizes for poor messaging. He claims that including the XBox One controller does adds very little to the cost of the Rift (it’s allegedly subsidized by Microsoft) and gives developers a common game pad to target until Touch is released.
When HTC announced the Vive pricing just yesterday — $799 including their touch controllers, it looked much better in comparison. If you look at social media- there are manly people claiming to be cancelling their Rift pre-orders for Vive.
Looking Ahead- Software
As we approach the dawn of virtual reality, we are starting to see some amazing software that is being developed. I could describe them all, but the clips below do the best job of explaining:
Looking Ahead- Hardware
I think that the future of virtual reality is in mobile. We aren’t there yet, but we’re making huge strides. The recently-announced Samsung Galaxy S7 puts VR at the front- even including a Gear VR with every pre-order. They have addressed the overheating issue by adding liquid cooling, which should be a huge help. Although we still can’t have the same quality of VR experience with mobile as we can with the Rift or the Vive, I think that this is the future. Of course desktop PCs will always be more powerful than smartphones (note to self: revisit this statement in five years) but the convenience and low cost of entry to mobile VR is what will take VR to the masses. People will bicker about the pros and cons of Constellation vs Lighthouse or the brand reputations of Facebook vs Valve, but I think that mobile should be the long term strategy for both. With so many players in mobile who could be quietly prototyping (cough *Apple* cough), it’s hard to say what mobile VR will look like even a year from now.
There are a couple of other interesting hardware players out there though. The first is Sony. They have a VR solution called Playstation VR (previously Project Morpheus) that looks like it could get a huge number of gamers into virtual reality. Little is known, but if they can get the price right, Sony could be the secret winner of the first major VR war.
Another interesting company is called Fove VR. They claim to have built the world’s first eye-tracking headset. Eye-tracking in interesting because it can be used in a technique called foveated rendering which will greatly reduce the processing power needed for VR.
The last topic that I want to touch on is social VR. Right now, there are two big players- AltspaceVR and VRChat. Both Altspace and VRChat are aiming to bring people together in a metaverse where they host regular concerts, meetings, parties, movies, and other events.
Another interesting project is Janus VR, which aims to bring the web to virtual reality. It’s really crude right now, but keep an eye on it- I think that Janus has potential.
A final note- Minecraft is coming to VR. Notch sold Minecraft to Microsoft, who is working with Oculus to bring Minecraft into VR. This could be the killer app for Oculus.
Well, my short post on the history of VR up to today has turned into quite the post. Hopefully you followed along for the ride, but I’m sure that the links and videos took you down some rabbit holes along the way. I am really excited to re-read this post in a year to see what has happened and how my opinions/predictions did.
Here’s a quick recap of what I think:
- Augmented Reality is the end goal, but we aren’t there yet.
- Mobile VR is the future, but the experience needs to be on par with what we can get on the desktop today. It’s coming soon though.
- The HTC Vive is awesome and targets gamers. It wins for hardware.
- The Oculus Rift is also aweseome and targets experiences. It wins for software.
- Social VR can enable some crazy stuff- we’ll see people working/communicating/living in different ways because of this stuff in the next 10 years.
Here is a video of what is now possible because of VR. The new Unreal Engine editor can put a developer into a virtual world to create their games. Creating VR inside of VR! Imagine all of the possibilities ahead of us. I’m really excited. Stay tuned to the blog for the more updates.