VR story — from a VR consumer and a WebXR perspective
Morpheus: Unfortunately, no one can be told what ‘The Matrix’ is. You’ll have to see it for yourself.
Intro
In times like these, we all look for something to pull our attention away from our remote work and the influx of news on the pandemic. We need a way to disconnect and relax.
Digital product sales are skyrocketing, which includes streaming movies and TV shows, but also games.
Even WHO recommends gaming as an effective way to deal with the reality of social distancing.
I also think it’s a great time to try virtual reality (VR) too. And I am not alone.
The new Virtual Reality game from Valve, “Half life: Alyx”, reached the TOP 25 in terms of popularity on the Steam platform, which is a historic event for the entire VR community.
Just one week before lockdown I bought my second VR headset and became an even more dedicated member of the small but strong VR enthusiasts community. And at the same time in Avenga’s Labs, one of the topics was to review the current state of in-browser VR and AR. Perfect timing.
Let me share my experiences with you, to help you understand more about the truths and myths of VR. This time it’s lightweight and more personal. Of course, there is a technical part about the new standard for multi platform VR development, called WebXR.
You can try becoming a VR fan, from the digital content consumer perspective. But also, if you are a developer, you can easily create VR apps using the tools you already know (JavaScript anyone?).
What is it?
There are different definitions. One of them is that every 3D rendered experience, like a 3D computer game or Google Street View, is VR. I take a more orthodox approach to VR and by that I mean, by wearing a headset it helps you immerse yourself in other worlds. The headset experience is key for the immersion.
Of course, there’s also Augmented Reality that renders 3D images and blends them with your physical world. The typical example of this are games on the top of your tabletop or educational applications for business. The easiest way to see it is to install AR games and experiences for your iPhone, as Apple invests a lot in this technology.
But that’s another article, as this one is not about blending real and virtual worlds but about leaving this world temporarily to experience another reality.
VR basics
Hardware
VR Headset — something that you put on your head with two lenses, one for each eye, rendering the 3D image and isolating you from the external world.
Controllers — the devices you hold in your hands that can represent, well, your hands in the VR world, but also other devices like control sticks, guns, light sabers etc. Controllers can also have different shapes and forms, for instance a VR gun.
Tracking devices — this can be a camera placed in front of the headset, tracking devices on the walls, or even playing with no tracking devices at all. In my opinion, tracking devices are one of the most common reasons people are reluctant to enter the VR world. It requires a physical setup that many people are simply not prepared for, plus the additional effort it takes can also be discouraging. As for me, I cannot even begin to elaborate more on that; for me, it’s always been a deal-breaker.
PC or Playstation 4 — the machine where the games and experiences are running.
Other Hardware:
I’m not going to discuss smartphones for VR.
There are also gloves to help you feel feedback from the virtual world, and treadmills to simulate walking and running, and many more.
How to start?
If you want to try VR, I recommend visiting someone who has a decent headset and playing with it for a couple of hours.
During the social distancing era, this is no longer a good idea.
Let me try to help you anyway.
How to start without killing the budget
Myth: VR requires an expensive PC with a powerful GPU, and complex setup with lots of cables, cameras, and sensors on the walls. Preferably all dedicated to the VR room with cables from the roof, etc.
True, if you want to experience absolutely the best quality with an expensive top of the line headset providing the highest quality image, wide depth of field, and high refresh rate. I’d call it a premium experience.
Unfortunately, it is still this common picture that discourages the majority of users from even trying VR. Plus the price of top-of-the-line headsets and a top PC can be as expensive as a decent used car.
The times have changed for the sake of easier adaptation.
Smartphone VR
Let’s start with the smartphone VR. It’s the cheapest option.
Even the limited cardboard experience shows what it looks like in VR 3D, and you have different images reaching your eyes and it does look 3D.
It works quite well but it’s not convenient.
You have to launch the VR experience, by putting your phone inside the cardboard device (the premium versions are actually made of plastic), you can watch it and interact with it. But, when you want to switch to another experience or to respond to incoming notifications on your phone, you have to take your ‘headset’ off, take your phone out, do your thing and then repeat the steps again to get back to VR.
Plus, there’s no real tracking of your movement. You are inside the scene and can turn your head around to see different parts of it and that’s it.
I tend to agree with the majority here, if you want to put in effort towards VR, don’t waste your time on a cardboard experience.
Non PC users
For XBOX users the topic ends here, no support for VR. Nothing is even planned for VR in future versions of the console.
Sony Playstation 4 (especially Pro) users are the ‘chosen’ ones here. There are hundreds of high-quality games, and the cheapest one comes bundled with a number of very good games plus controllers or virtual gun controllers.
The most price effective way to start a true VR experience is with PlayStation VR or a standalone Oculus Quest.
I can imagine here all the rants of PC users, that there’s Oculus Rift and other less expensive headsets. Yes, there are. But imagine the regular laptop users who don’t want to invest in a desktop PC or powerful gaming laptop, this guide is focused on them. For heavy PC gamers there are definitely more choices, with entire pricing and quality range from budget to high end.
PS VR
In the case of PSVR both the console itself and the VR set are often discounted and the total price is usually lower than the expensive VR PC headset itself (not mentioning the price of the PC). This was my first true VR experience, the difference from being VR ignorant to being a VR fan (albeit not fanatic).
Pros of PSVR:
- Broad set of high-quality games and experiences from PS Store
- Virtual gun for shooting games
- Very very comfortable headset with proper weight distribution
- Unique exclusive high quality PS games including multiplayer games for the PS platform
- High refresh rate (120 Hz)
- PS5 will introduce the new PSVR 2.0, so Sony will improve it further with the new generation
Cons of PSVR:
- Visible screen door effect — like an artificial wall between you and the rendered world, once you can see it, you cannot unsee it. It’s like watching a VR world except not being in it but seeing it through a virtual TV screen with a metal chain-link fence in front of it.
- Limited tracking (180 degrees) — your movements are tracked by a single camera looking at you from one direction.
- Cables — camera to console, an additional adapter, and a long thick cable to the headset itself
- Camera positioning and limited area — the message “out of area” happens too often even when you seem to nail it perfectly
- Low resolution and console performance
Wireless headset example
Wireless in VR technology simply means freedom; a lack of cables connecting the headset with other devices. In the extreme form, it’s a completely autonomous device that requires nothing else, like a console, PC, camera, or trackers, you simply just charge it and then put it on to start your VR experience.
There are others, certainly, but there’s so much love for Oculus Quest in the VR community. I decided to buy it and try it so as to have my own opinion about it.
What is interesting about Quest is the fact that over 90% of the buyers have never had any other VR headsets before. It’s really a big hit.
Oculus Quest
Pros:
- Immersion due to lack of cables and dependency on any other device, such as a PC or console
- Full 360 degrees immersion
- Tracking without sensors on the walls or an external camera
- If you have a gaming PC, you can connect Oculus Quest using Oculus Link and experience PC VR (with limited quality, but acceptable in my opinion)
Cons:
- Limited battery life (2–3 hours)
- Lower resolution and power due to a mobile chipset inside
- Despite the promise of freedom, it cannot be safely used outdoors (sunlight damages the screen)
If you don’t own a PS or PS Pro, the best entry to the VR world is Oculus Quest (or its future successor or competitors).
You need 2m x 2m (6’6” x 6’6”) of free space and . . . that’s it.
If you own a PS Pro it’s a more difficult choice. I’d still go with Oculus Quest as nothing beats 360 degrees of immersion.
I’m a gamer — what is there for me?
TLDR: Everything
All the genres are represented in VR. There are arcade games using your hands and entire body to interact with them, like the famous Beat Saber which uses light swords. FPS shooters have a completely different experience; they don’t point and click, or use the console controller to point at a target and squeeze the trigger.
There are puzzle games, RPG games and even RTS games. Games for younger audiences and hardcore horror games for those with the strongest hearts among us.
There are also ports of well-known PC games, and well-done ports are great. Simple ports show how different the VR design is from the classical 3D designs for the PC or console; Quake, Doom and other classics are there.
I’m not a gamer and never will be — what is there for me?
This is the part I’ll focus more on as it seems to be neglected all the time.
Explore the world (Google Earth VR, Wander)
Yes, you can travel all over the world and immerse yourself in famous locations, virtually any place in the world. You can revisit your favorite spots in a more immersive way than just looking at pictures. You can virtually be there, look around and move around.
Explore the universe
You can also experience the Apollo 11 mission, which is very impressive. Fly through the solar system and be the first person on Mars.
360 movies
So, 360 means the full immersion experience. There are a lot of really impressive experiences, like flying in a helicopter above the Alps, guided tours in different cities, and reaching places you’d never visit. There are experienced browsers downloadable as apps dedicated to VR. Additionally, you can use the VR version of YouTube to “watch” and experience a 360 3D movie.
By the way, you can also visit your own private cinema and watch movies in 2D in Netflix VR or watch with others at the same time on the BigScreen.
Relaxation
There are experiences like moving through a quiet forest or in the mountains. You can relax and meditate, so you can disconnect from the reality of confined spaces (especially in the times of quarantine). Beautiful graphics are augmented with relaxing music and the sound of the streams, wind, and rustling leaves. It’s truly hard to describe it in words.
Fitness
Box VR is a good example of an application to get you standing up and moving regularly inside your home when going outside is a risk, like in bad weather. Yes, you can exercise without it but the immersive experience works as a good motivator to spend 30–60 minutes exercising.
There are training sessions prepared by fitness experts and you can select your preferred training plans, etc.
Even if the app is not marked as a fitness app itself, there’s a lot of movement with VR compared to just sitting in front of a PC or console screen. For example, you stand up and use your hands, and you avoid obstacles by moving your head and entire body. Usually you don’t need to take a shower after playing console games, do you?
Paint
3D painting is really different from traditional 2D painting, as you create in a 3D space.
Known limitations
There’s no rose without . . . thorns. Honestly it’s good to know the limitations in advance to make more educated decisions.
Cables, sensors, and PCs…
Cables and sensors on the walls. Those limitations have stopped many people from entering the VR world altogether. But these can be avoided with an autonomous headset.
The need for a beefy PC is no longer required.
Walking in VR
One of the known limitations is that you can only walk freely in your safe area, let’s say 2 by 2 meters, maybe more depending on your location.
The workaround that VR apps use is the teleport. You select where you want to go and you appear there. Not very natural though.
Another approach is to imitate walking by using controllers. The field of view is limited and for most people, it’s not a pleasant experience. Your eyes see you moving but you don’t feel the acceleration and your mind will most likely protest against it. I’m the lucky one, as I haven’t experienced it ever.
In good VR apps there are settings for different comfort levels, so as to adapt your reactions to the movements in VR.
I’ve already mentioned treadmills and other solutions but they are not convenient (additional hardware) and don’t work so well.
In other words — you have to get used to it.
Legs tracking
Let’s take the example of a BOX VR fitness app that is about boxing in VR and tracking your hands to perform different exercises. It’s really great. But what about your legs — there are no sensors attached so they cannot be (easily) tracked.
Eye movement tracking
In the real world we don’t always have to turn our heads to change the point that we are looking at. In the majority of VR cases, you have to move your head to change the view. Of course, nothing restricts you from looking at the different parts of the scene, but you cannot realistically do it.
Battery life
Autonomous headsets require their own power source which is a battery. And of course, that means it’s limited. In the case of Oculus Quest, it’s around two hours of VR experience. For me it’s more of a blessing — it’s a natural kind of limit. I don’t have more time either way so no problem for me.
For those more dedicated, it’s possible to connect a power bank or even USB cable and continue for hours without taking it off (at the expense of lesser convenience).
Touch
Controller vibrations help a lot to simulate interaction with objects, like opening doors, but it’s not a real feeling of touching the object. There are special gloves, etc. but again, another hardware, another complication.
Multiplayer
There’s a common myth that VR is a lonely experience. It can be if you want it to be. But in general, that’s not true. There are VR chat rooms, and competitive and cooperative game modes. You can even experience the world in apps together with your friends.
Future
I believe the future lies in autonomous headsets without cables and augmented reality for smartphones and tablets. There are already announcements of new headsets coming out, in this category. New family of devices will arrive with better graphics (resolution, refresh rate) and performance, enabling even better experiences than today.
Why autonomous? (Again) Because they lowered the barrier by removing the need to have a powerful gaming PC. They are very convenient and really fun.
Developers, developers, developers — WebXR
For all the engineers, especially software engineers, there’s always these thoughts: “How can these things (games, experiences) be created? What are the Software Development Kits, environments, and toolkits to do that?”
And “What is an easy entry point to start playing with it?”
Let’s start with the gaming engines. Unreal engine, Unity and others have supported VR for a long time and are enabled to create dedicated, native, hardware optimized experiences for a wide range of devices.
There are of course plugins and even dedicated SDKs from all major hardware manufacturers like Facebook, HTC, Valve, Google, Microsoft, Samsung, HP and others. All of them enable developers to squeeze every bit of performance and capabilities from the device.
Important announcement here: at Avenga we do have a professional VR/AR team that works with all these complex frameworks and business VR/AR applications with a proven track record.
This time however it’s more like a test for the amateur. How difficult or easy is it to enter the VR world, as a creator using WebXR, which is supposed to work?
There are application stores with their requirements, reviews, and publishing policies for each VR platform out there. That topic is for another book, not this article.
A 3D content creation can be very expensive, and in extreme cases competes with Hollywood movies.
Let’s focus on something simpler then, easier to reach, and universal at the same time, like Cordova for mobile devices.
Short history
The first was WebGL, a technology that enabled the creation of 3D (rendered on flat screen) experiences and games inside a browser, without any additional plugins or extensions. It also meant integration with the power of local GPUs, common 3D models and object graphs.
The next move was WebVR. It has become relatively popular and was the true opening of VR inside the browser. Even today, it’s much easier to find WebVR content (usually with a message about it being deprecated) than the content from the latest standard, WebXR.
WebXR is the successor to WebVR. It’s a significant attempt to create better APIs and support for VR and AR on the web. There were major redesigns of the API and implementation, resulting in significant delays of the specification and implementation. The W3C organization finally accepted the specification.
How does it work from a user perspective?
You open a web page and see the Enter VR button or … see the message that WebXR is not supported.
But if you open it from the browser that supports it, the 2D web page will become a full VR experience, and all the browser controls disappear…
Are there emulators for developers?
Fortunately, yes. A WebXR device emulator extension is there to help you test WebXR solutions on your local machine. You can rotate your headset and press buttons on the controllers.
Putting on and taking off the headset all the time is of course very inconvenient. But as with every technology that creates at the abstraction level and common API, headset testing is a must, as usual.
It’s similar to mobile app development. Emulators help you a lot, but without testing on actual devices you may get an unpleasant surprise.
For instance, the demo app used to crash when I was pressing a button on the controller, when running inside the Oculus Quest headset. It never crashed in the emulator. Of course I figured it out with a trial and error approach.
Surprising lack of support
Sony Playstation VR doesn’t support WebXR and it doesn’t work at all; the same situation with Apple and its Safari browser.
It works however on MacOS with Chrome or Firefox (with an WebXR extension).
Apple is focused on AR instead using their proprietary framework on their hardware devices (a very Apple thing to do).
Proof of concept
My choice, based on research, was to try to use three.js to create a simple demo, build it locally and deploy it to the browser in my Oculus Quest and then prove it works.
There are other JS libraries and frameworks to do that, but this one seemed to me the easiest to start with. Additionally for me, not a daily front end developer, the vanilla JS approach was preferred over another framework, and on the top of the framework etc.
What I always do with any new tech or library, is that first I look at all the existing examples to familiarize myself with the capabilities of the library. Then I look at some real-world applications as more proof it might be worth trying. And finally I read the entire SDK documentation to figure out what the API capabilities are.
I have to admit the documentation is very good and the examples are very useful. I highly recommended trying it.
The library philosophy is that you have to build everything from an empty scene. Adding a camera that is looking at the scene and lights that cast light on multiple 3D objects are created using code.
The approach is similar to building an application when you start with an empty form and add various controls to it. This time however it’s not flat, it’s 3D, so there’s additional spatial imagination required.
And basic physics, like how the speed vector works and basic vector operations, are useful as well.
Basic concepts
Scene — something that you can see and is rendered in front of the headset; without the scene it’s an empty void. The scene has a background that has colors or textures.
Camera — a virtual device that is looking at the scene, with different fields of view (FOV) and angle distances. There are various types, and the one I used had a perspective view, which seemed more natural.
Lights — different types, colors and characteristics, that help to light the scene.
Geometry — representation of the shape of the object, like a box, line, sphere, or custom shape.
Material — representation of material from which the object is built, like basic materials or even physical materials with multiple textures. The more true to life a material is the more performance it demands.
An object is a Mesh of geometry and material. This is the way to create objects. Objects can also be imported from 3D models, but that isn’t the case in this specific Proof Of Concept (POC).
VR Button — it’s not just a button, it’s a VR enabler. It changes the mode of operations to VR, detects VR capabilities, and switches everything to the VR rendering mode. The Oculus Browser asks the user if VR should be enabled one last time before rendering an immersive VR scene.
The scope
The user is inside the futuristic wireframe and surrounded by boxes representing Avenga values, textures show the colored hashtag and a short description. Boxes are flying in the space and bouncing off the virtual walls. When the user presses the controller the boxes are launched from the controller’s position.
The demo is a mesh of several examples from three.js itself (thanks three.js team again).
You can see the movie recorded from the headset here:
And if you have a WebXR extension or even better, a WebXR capable headset, you can open this URL to experience it in true VR.
https://avengalabs-webxrvr-74a6c21.wao.zone
or
https://tinyurl.com/avengawebxrvr
The result
WebXR VR is working and it’s working very well!
Of course this simple demo doesn’t even touch on numerous other topics, but that wasn’t its purpose. Tinkering with it has become a joy in itself, but in the Proof Of Concept the important thing is to know when to stop.
How it is done (in three.js)
The code, as always, is the single source of truth and you can find it here.
Let me go through the workflow steps:
- Create scene
- Create camera
- Create lights
- Load textures
- Create geometries and materials (using textures)
- Create objects using geometries and materials
- Place the object in the right places (x, y, z) and rotation angles
- Create raycaster — ray of light for your controller (to select and interact with the objects)
- Add event listeners and functions to react to the controller actions
- Render function represents the classical game loop
- Evaluate the objects
- Move the objects using their velocity vectors and time passed since last updated
- Check for collisions (bouncing of the walls in this example)
- Calculate new vector for 3D velocity
- Render scene
Testing was done using a local web server (npm http-server).
Beware! Oculus Browser requires https, no way to go with an unencrypted http. Fortunately even a self-signed certificate has enabled testing on the headset.
Deployment for the world
Here at Avenga we do have internal/external CI/CD and hosting services based on our take on GitLab and Kubernetes.
The steps were following:
- Create a docker container using Alpine Nginx base to serve the static content.
- Connect to Avenga Kubernetes cluster and switch context.
- Create a pod.
- Expose a pod as a service.
- Create ingress to enable a dynamic link for the service.
- Done!
It was kind of fun (kubectl and docker commands) after some hours spent previously in JS on the client side.
The next step was to use wao.io (an Avenga product) to enable TLS, web caching and other performance and security optimizations, without touching the code or configuration. It was quick and easy.
Now it works with the proper TLS certificate and optimized delivery.
Final words
Virtual Reality was pronounced dead many times in the past. Now, even more manufacturers are announcing new devices this year.
Even with millions of active users, VR has not yet become mainstream and probably never will be. . . and it doesn’t have to be.
There are great experiences out there other than games, which are really enjoyable for different groups of users.
And if you ever become bored, and you happen to be a JS developer, there are easy ways to try to create something on your own.
And, for professional applications, Avenga can design and implement true business VR/AR applications that require much more skill, effort, and experience than this simple demo.
