Augmented Shared Reality in Unity

With Augmented Reality (AR) gaining popularity, tons of libraries and frameworks have popped up and a couple of them have managed to take the lead.

After developing a couple of AR and VR (Virtual Reality) experiences using Vuforia for Unity, I found myself with the challenge of developing a multi-user AR game. I did the first thing any developer would do: google to find someone who has done it before (don’t reinvent the wheel). To be honest, nothing I found was quite what I was looking for, and that’s why I’m writing this post- so you have better luck on your google quest.

I’m not going to teach you how to code in Unity nor how to use Vuforia , so you should understand the basics of both before reading on.

For the multi-user experience to work, I’m going to use the Unity built-in Network API. There are some other libraries out there, like uLink, that you can use to achieve a similar result.

Build the AR experience

First, let’s set up a scene in Unity, include Vuforia, and an ImageTarget with a 3D object of your choice. Fashback to the 90s (or 2016) and use a Pokeball™ as a target and Pikachu™ as object.

Now, it’s time to add some interaction for the user. As an example, I will let the user move the Pikachu™ around by dragging his finger on the screen. To do so, follow the basic script below:

You should make sure this is working fine on your device before moving forward.

Sharing the experience

Our goal is for multiple devices to interact with the AR scene created. To do this, we are going to use a host-client architecture: this means one device will be serving (and acting as a client, too) and the other devices will connect to it just as clients.

First, create an object with a NetworkManager component. This object will be handling the connections between the clients and the host.

There is a property called “Player Prefab,” this is the prefab that each client spawns when connected. We are going to spawn Pikachu™ for each player, so we should prepare our prefab.

Save Pikachu™ as a prefab and add a NetworkIdentity component to it (every object shared across devices needs an identity to be identified by both server and client). Since each client is going control their own Pikachu™, check the “Local Player Authority” box.

We should also add a NetworkTransform component; this is used because we want to synchronize the object’s transform (position in our case) on all devices.

We can now add our prefab to the NetworkManager. This also means we should remove Pikachu™ from the ImageTarget object, since now the NetworkManager will be in charge of spawning it.

There is one caveat though: the NetworkManager will spawn our object in the root of the scene, and we need it to be a child of the ImageTarget so that Vuforia can position it relative to it. A workaround is to attach a script like this to Pikachu™:

Don’t forget to add the tag “Stadium” to our ImageTarget (or any children) so that the script can find it.

Before trying this out, we need a way to tell a device to act as a host or client. We can add the built-in component called NetworkManagerHUD to our Network Manager object, this will display a basic UI on the device.

Now it’s action time: build the project and run it on at least two devices. On one of them, hit the LAN Host button, and on the rest specify the host IP (they need to be reachable from each other networks, or be on the same network) and hit the LAN Client button.

You will notice one undesired effect: when dragging Pikachu™ on a client, this will move all Pikachus™. This is because the DragMove script is attached to all Pikachus™ on the scene, no matter who “owns it”. To fix this, we need to move Pikachu™ only if the object is owned by the client:

Final Words

If you managed to follow the post, you should now get a basic idea of how networking works in Unity and how it can be combined with AR tools like Vuforia to generate unique experiences. As a the good book says: “Happiness is only real when shared”, so let’s build amazing experiences which connect people.

* Disclaimer: All Pokémon characters, regardless of which lingual name is in use belong to Nintendo, Creatures Inc, The Pokémon Company