How to record in Mixed Reality using SteamVR + LIV
You’ve seen those Beat Saber videos. They look absolutely incredible! Wait, are you telling us that you own a VR headset powered by a dedicated PC .. and you still haven’t tried capturing yourself in Mixed Reality?! Well, what are you waiting for, friend? Keep on reading!
Before we go on, let’s go through our shopping list of ingredients needed to make our Mixed Reality meal:
- A VR headset (really anything that is supported by SteamVR)
- A VR compatible computer
- Adequate lighting
- Space in your room to hook up your green screen and play area
- A webcam (an iPhone also works)
- GeForce Experience (or any other recording software of your choice)
Got all of the above? Fantastic sauce! Let’s get started.
Step 1: Calibrate your sensors for your respective hardware in your play area
Follow the instructions to calibrate your sensors of your respective VR device. Make sure after you calibrate that the play area can host the camera tracker. You don’t want the camera tracker outside of the sensor range otherwise this can’t work!
Step 2: Download Steam and install the required software
Download Steam on their website. After that, make sure to download the following:
(Optional Step): Calibrate your sensors in SteamVR
Open your Steam client, click on Library, then search for SteamVR Room Setup. Run the application and follow the instructions. Most people do this even after setting up their Oculus Rift, but we’ve had no problem at all skipping this setup after calibrating our Rift sensors.
Step 3: Download the OpenVR Advanced Settings exe
You can find it here.
After you follow the top 3 steps successfully, kindly restart your computer.
Step 4: Launch LIV and install the SteamVR Driver
When you launch LIV successfully, click on the Install button. This will install the SteamVR driver that is required to track the camera that you will use to capture your awesome gameplay footage!
If all is well, you will see a new friendly component in your suite of SteamVR items:
Step 5: Edit camera settings
In your LIV application, click Launch Compositor. Wait for the application to load, then click Camera. After that, click the add camera icon.
Camera detection
In the Edit Camera menu, click the Type dropdown and search for your webcam configuration. If all goes well, you will find it in the list! Double click it twice and wait for it to load in the LIV Output Window.
(Optional) iPhone setup
Download the LIV Camera App from the AppStore. Make sure you are connected to the same network as your computer. On the LIV App on your PC (in the edit camera settings), head on over to the Camera and select the Type as LIV Camera for iOS (NDI) as shown below. Click the refresh icon (with your LIV Camera App open on your iPhone).
If all goes well, you should see your phone. Double click on it to see your camera output.
Keying
Keying is quite simple but it does require a little bit of patience. First, pick the color that you’d like to key (if you have a blue screen, then go with that color). To get the exact color, what we’d recommend you do is to take a photo of the green screen (or whatever color), upload it on your computer, then use the eye drop to get the exact hexadecimal. Then, when you get the hexadecimal, input it in the Color option.
Next, you’ll want to adjust the Threshold and Smoothness until you get the desired output. Make sure you are standing in front of the camera anytime after you change the values so that you not only get the output correct (all black), but also you the player are captured well with the background.
Calibration
Quite possibly the most tedious part is the calibration. Remember the patience you had before for keying? Well, if you want to get the calibration right, you should definitely have more patience for it! Just to be clear, you aren’t calibrating your controllers: you are calibrating the camera.
Once you put the camera at a position where it’s inside the tracker (around 1.5m away from the player) and at an angle where it captures the player well, you can begin calibration. Choose a tracker of your choice (left controller or right controller). Then follow the onset instructions to calibrate the camera tracker:
- For the first calibration point, the controller tracker must be touching the camera (almost kissing it)
- For the second and third calibration points the user must stand at least 1.5m away and calibrate opposite corners of the room (upper left and bottom right)
If all goes well, you should see your controller as a 3d object layered on top of the real controller object! Refer to the gif below for a better understanding.
If you still feel that your camera tracker can be optimized better, refer to ragesaq’s message below on the LIV Discord (it’s the 6th pinned post in the #🚧general-support🚧 channel).
Whew, the hardest part is now over. Now, let’s have some fun!
Step 6: Sync and Launch a game
Before we start having fun, make sure that the game you want to play is on Steam and it’s supported by LIV. Here are a list of games if you want to get an idea on what you can play while capturing yourself.
Head on over to the Capture section and click the drop down for the apps that are detected by LIV and are compatible. Click Sync & Launch to kickstart the game. Have fun!
Step 7 (bonus): Sync and Launch your Unity VR game
Yes, you can most definitely test your video game in Mixed Reality as well! First, download the LIV SDK. You must fill out a form and submit it. After that, someone will e-mail you with the package.
Next, open up your Unity project and import the LIV SDK package you just downloaded. Create an empty GameObject, call it LIV, click Add Component and search for LIV.
Since we are using SteamVR, we want to make sure that our Tracked Space Origin has the Player prefab selected and our HMD Camera has the VR Camera from the SteamVRObjects child GameObject from Player selected.
Now, head on back to the LIV app on your PC. In the Capture section, head on over to the Manual tab, check Use SDK and change the target to your Unity target. All you need to do now is go back to Unity and click Play.
Ta-da! Hope you enjoyed this tutorial by WeAre Studios
