How to Embed Group Video Chat in your Unity Games

Author Note: this publication is the first part in a set of Agora demos you can find here, where I show you how to implement video party chat, call stats UI, live audio broadcasting, and spatial audio. Cheers!

When you’re making a video game, you want to squeeze every last drop out of performance out of your graphics, code, and any plugins you may use. Agora’s Unity SDK has a low footprint and performance cost, making it a great tool for any platform, from mobile to VR!

In this tutorial, I’m going to show you how to use Agora to create a real-time video party chat feature in a Unity MMO demo asset using the Agora SDK and Photon Unity Networking (PUN).

By the end of the demo, you should understand how to download the Agora plugin for Unity, join/leave another player’s channel, and display your player party in a scalable fashion.
For this tutorial, I’m using Unity 2018.4.18.

Before we get started, let’s touch on two important factors:

Since this is a networked demo, you have two strategies for testing:
1. Use two different machines, each with their own webcam (I used a PC and Mac laptop)
2. From the same machine, test using one client from a Unity build, and another from the Unity editor (this solution is suboptimal, because the two builds will fight for the webcam access, and may cause headache).

Getting Started

• Create an Agora.io developer account here to get your AppID
• Create a Photon developer account for their appID
• Import Agora SDK into your project from the Unity Asset store
• Import Photon Viking Multiplayer Showcase into your project

After importing the Viking Multiplayer asset, the PUN Wizard should automatically display. If it doesn’t you can find it in Window > Photon Unity Networking > PUN Wizard > “Setup Project” > Paste your AppId or email you used for your Photon account.

Create Agora Engine

Our “Charprefab” is the default Viking character we will be using. This object lives in Assets > DemoVikings > Resources.
It is already set up with Photon to join a networked lobby/room and send messages across the network.

Create a new script called AgoraVideoChat and add it to our CharPrefab.
In AgoraVideoChat let’s add this code:

Note: This project is meant for reference purposes and development environments, it is not intended for production environments. Token authentication is recommended for all RTE apps running in production environments. For more information about token based authentication within the Agora platform please refer to this guide: https://bit.ly/3sNiFRs

This is a basic Agora setup protocol, and very similar if not identical to the AgoraDemo featured in the Unity SDK download. Familiarize yourself with it to take your first step towards mastering the Agora platform!

You’ll notice that photon.isMine is now angry at us, and that we need to implement some Agora callback methods.
We can include the proper Photon behavior by changing
public class AgoraVideoChat : MonoBehaviour
to public class AgoraVideoChat : Photon.MonoBehaviour .

Agora has many callback methods that we can use which can be found here, however for this case we only need these:

Let’s play our “VikingScene” level now, and look at the log.
Huzzah!

We are in an Agora channel with the potential to video-chat 16 other players or broadcast to around 1 million viewers!
But what gives? Where exactly are we?

Create Agora VideoSurface

Agora’s Unity SDK uses RawImage objects to render the video feed of webcams and mobile cameras, as well as cubes and other primitive shapes (see AgoraEngine > Demo > SceneHome for an example of this in action).

1. Create a Raw Image (Right-click Hierarchy window > UI > Raw Image) and name it “UserVideo”
2. Add the VideoSurface script to it (Component > Scripts > agora_gaming_rtc > VideoSurface)
3. Drag the object into Assets > Prefabs folder
4. Delete the UserVideo object from the hierarchy (you can leave the canvas), we only wanted the prefab

5. Add this code to AgoraVideoChat

6. Add the newly created prefab to the UserPrefab slot in our Charprefab character, and uncomment CreateUserVideoSurface() from our callback methods.

7. Run it again! Now we can see our local video stream rendering to our game. If we call in from another agora channel, we will see more video frames populate our screen.
I use the “AgoraDemo” app on my mobile device to test this, but you can also use our 1-to-1 calling web demo to test connectivity, or connect with the same demo from another machine.

We now have our Agora module up and running, and now it’s time to create the functionality by connecting two networked players in Photon.

Photon Networking — Party Joining

To join/invite/leave a party, we are going to create a simple UI.
Inside your CharPrefab, create a canvas, and 3 buttons named InviteButton, JoinButton, and LeaveButton, respectively.

Next we create a new script called PartyJoiner on our base CharPrefab object. Add this to the script:

• Add the corresponding “OnButtonPress” functions into the Unity UI buttons you just created.
  [Example: InviteButton -> “OnInviteButtonPress()”]

• Set the CharPrefab tag to “Player”
• Add a SphereCollider component to CharPrefab (Component bar > Physics > SphereCollider, check the “Is Trigger” box to true, and set it’s radius to 1.5

Quick Photon Tip — Local Functionality

As you can see we need to implement two more methods in our AgoraVideoChat class. Before we do that, let’s cover some code we just copied over.

“If this photon view isn’t mine, set my first child to False” — It’s important to remember that although this script is firing on the CharPrefab locally controlled by your machine/keyboard input — this script is also running on every other CharPrefab in the scene. Their canvases will display, and their print statements will show as well.
By setting the first child (my “Canvas” object) to false on all other CharPrefabs, I’m only displaying the local canvas to my screen — not every single player in the Photon “Room”.

Let’s build and run with two different clients to see what happens…

…Wait, we’re already in the same party?

If you remember, we set private string channel = "unity3d" and in our Start() method are calling mrtcEngine.JoinChannel(channel, null, 0); . We are creating and/or joining an Agora channel named “unity3d”, in every single client right at the start.
To avoid this, we have to set a new default channel name in each client, so they start off in separate Agora channels, and then can invite each other to their unique channel name.

Now let’s implement two more methods inside AgoraVideoChat:
JoinRemoteChannel(string remoteChannelName) and GetCurrentChannel() .

This code allows us to receive events that are called across the Photon network, bouncing off of each player, and sticking when the invited Photon ID matches the local player ID.
When the Photon event hits the correct player, they have the option to “Join Remote Channel” of another player, and connect with them via video chat using the Agora network.

Test the build to watch our PartyJoiner in action!

Finishing Touches — UI + Leaving a Party

You have now successfully used Agora to join a channel, and see the video feed of fellow players in your channel. Video containers will pop in across your screen as users join your channel.

However, it doesn’t look great, and you can’t technically leave the channel without quitting the game and rejoining. Let’s fix that!

UI Framework

Now we’ll create a ScrollView object, to hold and organize our buttons.

Inside of Charprefab > Canvas: make sure CanvasScaler UI Scale Mode is set to “Scale With Screen Size” (by default it’s at “Constant Pixel Size” which in my experience is less than ideal for most Unity UI situations)

• Inside our CharPrefab object, right-click on Canvas, select UI > Scroll View

Make sure your Anchors, Pivots, and Rect Transform match mine

• Set “Scroll View” Rect Transform to “Stretch/Stretch” (bottom right corner) and make sure your Anchors, Pivots, and Rect Transform match the values in the red box pictured above.
• Uncheck “Horizontal” and delete the HorizontalScrollbar child object
• Set “Content” child object to “Top/Stretch” (rightmost column, second from the top)

• Create an empty gameobject named “SpawnPoint” as a child of Content — Set the Rect Transform to “Top/Center” (Middle column, second from the top) — and set the “Pos Y” value to -20
  Make sure your Anchors: Min/Max and your Pivot values equal what is displayed

In AgoraVideoChat add:

In Start() add playerVideoList = new List<GameObject>();.

We’re going to completely replace our CreateUserVideoSurface method to:

and add two new methods:

Comment out the UpdateLeavePartyButtonState() for now.
Next, drag in your newly created ScrollView UI objects into the appropriate slots, and set their rect transform values like so:

Almost there!
Now all we have to do is add the methods for “Leave Party” functionality in AgoraVideoChat:

and update our AgoraVideoChat callbacks:

and in PartyJoiner:

Play this demo in two different editors and join a party! We start off by connecting to the same networked game lobby via the Photon network, and then connect our videochat party via Agora’s SD-RTN network!

In Summary

• We connected to Agora’s network to display our video chat channel
• We enabled other users to join our party, see their faces, and talk with them in real time
• We took it one step further and built a scalable UI that houses all the people you want to chat with!

If you have any questions or hit a snag in the course of building your own networked group video chat, please feel free to reach out directly or via the Agora Slack Channel — https://www.agora.io/en/join-slack/

Check out the link to the full project:
https://github.com/joelthomas362/agora-partychat-demo