How to Embed Drag and Drop Live Video Streaming into VR (Oculus Quest)

Who said teleworking had to be boring?! Ever wanted to chat with your friends or see what’s happening on the other side of the world while in VR? What about a VR trade show where you are able to stream straight from businesses to your VR app? Interested in getting up and running with Oculus Quest but not sure where to get started? Here is a quick guide teaching you how to integrate Oculus Quest and create a drag and drop solution for Agora.io’s Realtime live video streaming backed by their amazing subsecond latency global network.

Install Fest

This one is going to be short, all you need is the

• Unity Editor (with Android Support)
• A developer account with Agora.io
• Oculus Developer Account
• Oculus Quest

Getting Started

To start, open Unity and create a blank new project named Agora Quest Demo.

Switch Build Platform

To save time and as we currently know, we are going to be building for Oculus Quest so we can go ahead and change the Platform to Android by going to File/Build Settings and choosing Android in the Platform section.

Also, Make sure you set your Texture Compression to ASTC.

Install Fest

Next, navigate to the Unity Asset Store (if you are in the scene view, click the Asset Store tab), Import and Download two assets “Oculus Integration” and “Agora Video Chat SDK”. Oculus takes care of a lot of the heavy lifting of getting started in VR and Agora has your back when it comes to global communication.

While downloading the Oculus Integration package you will be prompted to update your Spatializer Plugin and Restart your Unity Editor, don’t panic, just hit Ok and Restart when they pop up.

Modify Project Settings

First, we need to change a few parameters in the Player Settings, so now go into your File/Build Settings/Player Settings and in the Inspector, at the top update your Company Name and Product Name.

Next, go to your Other Settings and Deselect Auto Graphics API and also get rid of Vulkan.

Now change the Package Name to match your Company and Project you set earlier and your Minimum API Level to API Level 21.

Last, scroll down to the bottom to XR Settings. Select Virtual Reality Supported and under Virtual Reality SDKs select Oculus.

Make a Scene

This time we won’t quite be making a scene as much as we will use a premade scene from Oculus found at Assets/Oculus/VR/Scenes/Room and modifying it to fit our needs. Go ahead and open up the Room Scene. As you can see there is an empty room with light decoration and some simple hand geometry. A perfect canvas to start with as we are focusing on getting communication running. I will remove a couple of walls in a feeble attempt to make the scene my own and then create an Empty GameObject to set as a location placeholder for my video screens. I name this object VideoSpawn and move the position right in front of the camera at a Z position of 1.92.

Create an Agora Drag And Drop Prefab Instance

Next up we will need a way for our app to talk to the agora network. The fastest way to do that is to Create an Agora Prefab by creating an Empty GameObject in your scene and name it “AgoraInstance”. Next, take the code snippet below and add it to the new AgoraInstance.

You can now drag this into your hierarchy to save it as a Prefab, Anytime you drop this into a scene it will allow you to automatically create an agora instance and join a scene when the Prefab is activated.

Now in your Inspector you can add your Agora.io App ID and whatever room name you want to test in.

Modify Android Manifest

Next, you will need to modify your Android Manifest file. Because the Oculus Quest doesn’t have cameras (well, none that can be used for this demo) we will get rid of the camera use requirement ( <uses-permission android:name=”android.permission.CAMERA” />) located on line 9 in the folder Assets/Plugins/Android/AgoraRtcEngineKit.plugin

Test on Device

Now you will need something to test off of, you have tons of options here as Agora is not Unity specific. If you wanted you could just create a sample agora video scene if you are also on a Mac you can learn how to do that here or Windows here. But, to be fancy we are going to broadcast from a web app running on a Heroku server (Shout out to Hermes for creating it!). Click this link, add your App Id and the room name and hit join, halfway done!

Now that you have some live video coming down the pipe go ahead back into your Unity Editor and select File/Build Settings. Drag and drop your modified Room scene into your Scenes to Build section and hit Build and Run.

After a short loading sequence, your app will deploy to your Oculus Quest Headset. Congratulations! If you want to run your app from within the Oculus headset without rebuilding, inside your headset go to Library/UnkownSources and you will see your app. Great Job!

Other Resources

• The Agora.io Video SDK for Unity available on the Unity Asset Store
• Get the Source Code for the web app here.
• The complete API documentation is available in the Document Center.
• For technical support, submit a ticket using the Agora Dashboard.
• The complete Oculus Documentation is available here.