Graphical Telnet
Late last year I was considering how to add multiplayer to my game Pathos. There was over 200K lines of code so rewriting to introduce a typical networking abstraction would be far too time consuming. I needed a lateral solution that would preserve my code investment. How do you turn a single-player game into multiplayer without starting from scratch?
I decided to just get started and build a split-screen mode. This allowed me to sort out a number of code problems related to having more than one player. This was an relatively easy refactor and works well if you don’t mind getting cosy with your BFF on a tablet.
The next step was to explore screen-sharing software such as TeamViewer. This meant the split-screen game could now be played over the internet. It was fun to see the other player in action but it did cost half of the screen real estate. But the main limitation was the lack of support for two mice at once. In the end, I did not find a technology or product that worked well in this space.
I must segue to explain that my game is built on an user interface abstraction library called Invention. This was an early decision to build native apps for Android, iOS and Windows but still have proper code sharing. With thanks to C# .NET, Mono and Xamarin this project was going great.
The modest success with TeamViewer had me thinking how I could write my own remote projection code. It irritated at me for several weeks before the breakthrough realisation:
I can use the same techniques and build a remote platform as a sibling to the native platforms!
I named this remote platform ‘Server’ and I’ll do my best to explain how it works. It is both amazing and amazingly difficult to explain.
The application code runs entirely server-side. The server listens for clients to connect via TCP/IP sockets. There is a separate instance of the application for each client, hosted in the server. The client is thin and maps the server messages to native controls and has no specific application logic. What we are talking about is Graphical Telnet.
The application is running in the server and the user interface is projected down the socket to the client app. For the end user, the remote app behaves like any other native app.
The server platform is also portable code which means you can host in Android, iOS or Windows. It’s not peer-to-peer but you don’t need a central server either. You can host a game on your iPhone and your friend can join with their Android tablet over your local WiFi or even the internet if you don’t mind some firewall port forwarding.
The server state is easily shared between the client applications because it is just objects in memory. There is no inter-process communication or messaging protocols to be handled by the application code. You can even stick to single-threading to avoid the hassle of critical sections.
The communication protocol uses a compact binary format similar to the .NET BinaryWriter/Reader API. The actual sending and receiving of packets is via asynchronous queues. Assets such as images and sound effects are sent to the client as they are needed and then cached.
On first connection, the server performs a version handshake to detect breaking changes in the protocol. Incompatible clients are immediately disconnected and must be upgraded. Conceivably, you could deploy a generic client app (ie. Graphical Telnet Client) to connect to any Invention app running as a server.
Delta changes to the user interface are collated into packets and sent to the client, up to 60 times a second. Nothing is sent if there are no changes to the user interface on the server.
There is an important limitation of this remote platform. If your app has animations at 60 FPS then we need to be able to send a packet every 16ms. This means the tolerable latency is <32ms between the client and server. If latency exceeds this maximum then there will be lost frames and lag spikes. Of course, a less intensive app will cope with much higher latency. As good or better than most web apps anyway!
Let’s consider an example of a button that changes from grey to blue when it is tapped by the user.
var Surface = Application.Window.NewSurface();var Button = Surface.NewButton();
Surface.Content = Button;
Button.Alignment.Center();
Button.Margin.Set(5);
Button.Padding.Set(5);
Button.Background.Colour = Inv.Colour.DimGray;var Label = Surface.NewLabel();
Button.Content = Label;
Label.JustifyCenter();
Label.Font.Size = 30;
Label.Font.Colour = Inv.Colour.White;
Label.Text = "CONNECT";Button.SingleTapEvent += () =>
{
Button.Background.Colour = Inv.Colour.DodgerBlue;
Label.Text = "CONNECTING";
};Application.Window.Transition(Surface).Fade();
The basic user interface looks like this in the two states:

It may help to think of the communication protocol as a continuous stream of user interface updates. A bit like characters and line feeds in the telnet protocol. The follow is a text representation of the messages as separated by packet boundaries (==).
== SENT TO CLIENT / RECEIVED FROM SERVER ==
var S1 = Application.Window.NewSurface();
var P1 = S1.NewButton();
P1.Alignment.Center();
P1.Margin.Set(5);
P1.Padding.Set(5);
P1.Background.Colour = Inv.Colour.DarkGray;
var P2 = S1.NewLabel();
P2.JustifyCenter();
P2.Font.Size = 30;
P2.Font.Colour = Inv.Colour.White;
P2.Text = "CONNECT";
P13.Content = P14;
Application.Window.Transition(S1).Fade();== RECEIVED FROM CLIENT / SENT TO SERVER ==
P1.SingleTap();== SENT TO CLIENT / RECEIVED FROM SERVER ==
P1.Background.Colour = Inv.Colour.DodgerBlue;
P2.Text = "CONNECTING";
The client reads the messages from the server and updates the user interface to match. When the end user taps a button, the client sends these messages to the server. The server responds to these messages by executing application code which results in more user interface updates and further messages sent to the client.
There are a more details to this Server platform that I won’t go into in this article. For example, there are provisions for the server to ask the client to invoke a native API such as launching the default email app.
This approach is definitely not suited to every kind of multiplayer game. But when you are developing in Invention you get graphical telnet multiplayer for just about free. Writing multiplayer apps can be difficult when you need to keep the server and client state and shared and synchronised. Graphical telnet makes this as easy as writing a single player game.
