Multi-Screen Physics using Web + Native
Connecting mobile devices over WebSockets is something we’ve done for client projects such as Racer and Penguin Curling. We wanted to take this idea further and do it over a local network using local WiFi Peer-to-Peer connections available on Android.
The Set Up
We used our native wrapper on Android to house the application, as well as a fullscreen webview to render a WebGL scene with the PixiJS library. The scene itself is simple, consisting of just a number of circles that behave with physics and fill the screen.
Physics
Our internal framework, Hydra, has a built in particle physics engine, which allows us to easily create the simulation and attach behaviors such as collision, and repulsion in the case of pushing objects away from the mouse.
We create 750 circle sprites in PixiJS and attach each to a Particle object which is used inside the physics simulation as an abstract reference and then copies its position back to the sprite, moving the circle on the screen.
Connecting Multiple Devices
While researching, we found Android has something called WiFi Peer-To-Peer which allows different devices on the same network to discover each other in order to communicate.
Even though two devices are connected, WiFi P2P only acts as an initial handshake giving you the IP address of the other device. From there, we had to create a socket connection so we can pass data. For this, we used a library called ASyncSocket.
With all the Java code written, we could now create a JavaScript class to communicate with the native code, utilizing Hydra’s native bindings.
One step that we learned while building Racer was that we need a reference in time in order to calculate latency between devices. This means that when Device A receive a message from Device B, Device A will know the message took 20ms to be received and can manage on-screen graphics accordingly.
When a device is paired with a new connection it will send pings to that device 11 times, each time calculating how long the round-trip took. Once it has all 11 responses, it chooses the median time and that value is used.
Putting it all together
Once two devices are paired, the application begins working in multi-screen mode. On the top screen, when a user touches an event is sent to the bottom device with the screen co-ordinates of the touch. The bottom screen also uses the latency value to calculate what the position and velocity of the circle should be, given the difference in time from when the touch happened.
