Welcome To The Holodeck, Lets Get Started
When you are goggled in the environment you are standing in is seemingly supplanted by a virtual one. If the virtual environment is bigger than the real one and you walk towards the horizon, you’re going to hit a couch/wall/cat and fall long before you reach the Lich King’s lair.
As mentioned in the last post, one method for controlling your avatar in the virtual world involves optical sensors like the kinect or Leap Motion to create what we really all want: a holodeck in our living room.
Some people, like VR researcher and boundary-pusher Oliver Kreylos, and the guys over at Survios, are already hard at work bringing this technology to life. What I’ve been thinking about is how the experiences will work and what can be built on the platforms they are developing.
The Set Up
You’re going to need space. Just like any motion game from knect to dance dance revolution, this type of virtual reality experience will require a little more room than a traditional controller. Your holodeck is basically a taped off square with a sensor in every corner and a few feet of open area around it. The sensors cover a conical space in front of them like a projector would which creates a circular area of usable space in the center of the square. That extra space outside the square is a buffer zone for safety, more on that later.
Each one of the sensors serves up 3 kinds of data:
- Traditional color photo bitmap data.
- Depth data: how far away from the sensor each pixel of the photo is.
- Skeleton data: where every major joint in your body is in 3D space.
Using a laptop, the data from all 4 sensors can be blended and wirelessly streamed to the headset on the player. VR headsets know which way your head is pointing, and the sensors know where your head is, so together you can move and look around freely. This is a 1-to-1 mapping so moving one foot in the real moves you one foot in the virtual as long as you stay within the taped off area.
Exit Detection
Moving freely is great, but the taped off area is pretty small when compared to the infinite spaces possible in virtual reality and common in gaming experiences. One of the neat things about tracking your skeleton is that the system knows if you have left the area. If it can’t detect your body, then you must be out-of-bounds.
Besides mobility, price, and audience, android-powered VR headsets have one more advantage over the Oculus Rift: a camera. A few other headset manufacturers (like vrVana) have begun adding cameras to their rigs, which is a good call because it’s going to be important.
If and when the player steps out-of-bounds, the system will know immediately and should take action to ensure the safety of the player. If your headset has a camera on it, the blindfold of the virtual can be lifted to reveal live video of what’s actually in front of your face. Essentially, sticking your head out of the active area will be like sticking your head back into the real world.
Just because you've left the game doesn't mean the game has left. In Mario Kart 64, if you drive off the edge of the road, Lakitu comes and takes you back to the game. If you exit the active area, augmented reality could be used to draw a similar character over the real world, directing you to turn around and step back into your game.
Coarse and Fine Movement
Being able to move freely in a 1-to-1 mapped virtual space is amazing, but it’s also very limiting because the real active area is just a few feet in diameter. Plenty of experiences can be had in a 10 foot bubble, but exploring a vast virtual world is a tremendous draw for VR.
In order to move around a world, the player needs to be able to move virtually while standing still in the real. If the player is able to switch between movement modes they effectively gain the power to move their 1-to-1 movement bubble around in the virtual world much like you might pick up and move a mouse when it reaches the edge of a mousepad.
Fine movement
- Player movement 1-to-1 mapped
- Hands are used for interacting with objects
- Good for moving around small spaces
Coarse Movement
- Player stands still and navigates with gestures
- Hands are free to control speed or other actions
- Good for moving around large spaces
VR experiences can contextualize the fine and coarse movement modes as walking around vs riding some vehicle. A fun example I’ve been imagining is an Aladdin Cave of Wonders game where you navigate between fine movement puzzle rooms using coarse-movement via the magic carpet.
Mechanics of Coarse Movement
One way to control the coarse-movement is by way of what I call the “Volumetric D-Pad”. The D in D-pad stands for directional, a D-pad is the plus shaped set of buttons on a videogame-controller. The player would let the system know to switch movement modes by gesture or voice or some in game button. Once in coarse-movement mode, their body acts like a thumb on an invisible D-pad. If you step forward in the real you move forward in the virtual and keep moving forward until you step back into the neutral center of the virtual D-pad.
One interesting advantage of this approach is that because the headset turns your view automatically, there’s no need to reset the D-pad to change directions, if you turn to your left and step forward you will move “left” but it will just feel like forward. An extension of this idea leads to the next logical development: the “Volumetric Joystick”.
The joystick is very much like the D-pad, but rather than having discreet volumetric “buttons” for the cardinal directions, it allows the player to control speed and direction fluidly by moving away from a center point. Essentially, when you switch to coarse movement mode, the point you’re standing on becomes the center of the joystick. If you step to the right at a 73 degree angle, you move in that direction. The further from the center you move, the faster your movement.
With these mechanics, it’s possible to create a wide variety of virtual reality apps and games using hardware that is commercially available right now. Like any new medium, the limitations will shape the experiences.
Come back next time to learn about how technology will shape VR narratives.
