Solving WebSockets & Networking Issues in Magic Leap (Unity3D)
UPDATE: Please refer to this blog post for more details, I should have posted this sooner. Also, the solution proposed in this medium post only works for UDP socket and not TCP/IP (Hence, it doesn’t truly solve websockets).
If you want to cut to the chase just jump to the Solution section.
Around early-August last year Magic Leap One: Creator edition was finally announced. Offering a better field of view, larger compute (because of the Lightpack) and a handy controller with 6DoF tracking, it opened avenue for a large number of applications.
Since I have been working at company in San Francisco (one of the first 6 cities to which magic leap provided shipping), I was lucky enough to test one of these early on when the company bought couple of them. When I started developing applications on it as part of my work, one of the challenges I faced was to get networking and websockets working. The main issue was that everything seemed to work well in Zero Iteration but when I deployed it on magic leap in form of an mpk, it stopped working. I finally figured out the solution after receiving some help from Shane Engelman of Magic Leap. Since, I couldn’t find any documentation related to this, I decided to write this medium post to help others who have trouble getting networking to work.
Why networking?
A lot of potential and hype in mixed reality is because it allows for shared experiences. The hope is that at some time in future, we would all be wearing mixed reality headsets and tuning into channels (similar to a TV channel) to watch shared experiences.
Even without all that, many simple mixed reality applications require communication between the headset and internet or maybe some other device (Android/iPhone/Magic Leap/Hololens) on the same/different network. Hence, getting networking to work is essential for this.
The Problem
So, the main problem I faced while solving this issue was limited documentation on magic leap website. So, when networking wasn’t working I could think of two possible issues:
- Maybe “System.Net.Sockets” wasn’t compiled for the leap. For instance, in case of Android, you can’t just use .Net code in Unity and try building applications unless it has been cross-compiled for Android in some way.
- The other possible issue could be a simple permissions issue. Most platforms like Android and iOS have permissions and the default settings usually prevent access to most privileges like internet etc.
So, turns out the issue was just a permissions issue. It had to be set in a specific way by assigning an internal script to a game object to give internet and socket privileges. I found the solution after Shane pointed me to this post from magic leap developer forum. As per the post, official support for WebSockets in Magic Leap would be provided in the 2018.3 release.
Solution
So, I am assuming here that you are on a Magic Leap version of Unity, have your project all set up and only have issues with networking. If you want help setting up your magic leap Unity project, then I would recommend googling “Magic Leap Unity tutorials”. There are plenty of medium posts and blogposts about this.
So, once you have your project set up for deployment, you just need to follow the following steps to get networking working:
1) In your Assets directory, find the PrivilegeRequester.cs script. It should be located in the Assets/MagicLeap/CoreComponents directory. Now, drag this script onto any active GameObject in your Unity Scene (Note that this game object needs to be active when your Unity application is deployed onto the magic leap).
Once you have attached the script, increase the Privileges size to 1 and select Local Area Network from the Element 0 dropdown.
2) Go to File -> Build Settings -> Player Settings -> Publishing Settings and ensure that you at-least have Internet and LocalAreaNetwork Privileges selected. Note that depending upon your project you might have other permissions selected too.
3) Connect Magic Leap to your desired Wifi Network following the instructions here.
Just doing the above three steps should ensure that your scene is ready for networking. Now, when you deploy your application onto Magic Leap, after the loading screen, you should see a popup with a message like:
“Allow the_app_name to access Local Area Network” -> Accept/Reject
Once you select “Accept” from the popup, your networking code should be working. In my case, I was trying to get an iPhone to send some messages via UDP socket to a magic leap on the same wifi network and I can confirm that doing the changes listed above makes it to work. Basically, enabling LAN in PrivilegeRequester.cs script results in a function call to the internal RequestPrivilegeAsync function which results in the above popup.
Debugging
I would say the best way to debug networking is to test if your code first works in Zero Iteration. If it works in Zero Iteration, then after making the changes listed above and deploying it to magic leap, networking should work.
You can also debug your magic leap application by using mldb via Wifi. The details about this can be found here.
UDP Socket Programming Note
To implement any kind of socket functionality, you can use the functions from System.Net.Sockets and System.Threading classes. There are plenty of examples online which illustrate how to do this like this Unity3D forum post.
Initially, I wanted to make an example application in this post to illustrate the socket functionality, but the post seemed to be getting too long, so I just included the instructions on solving the main networking issue. If you have any simple application in mind which would be cool for displaying networking functionality, please mention it in the comments below. If I get time, I would be sure to develop one of them and make a post about it.
Conclusion
Hope that this post was useful for you and helps in solving your issues :-).
If you have any queries/feedback feel free to leave a note in the comments below!