Under The Hood: Simplay’s PHOTON Technology

A closer look into the technology that makes Simplay tick

Earlier in 2018, our newly developed propitiatory streaming technology has finally been released to the public. Dubbed “Photon”, it aims to improve picture quality, reduce input lag, and ensure the overall user experience remains as seamless as possible; ideally, it would be as if your Cloud PC was sitting right there next you.

Today we’d like to take a few moments to talk about what goes behind this technology, and what makes it so awesome. Before we dive in, however, let’s start from the beginning.

What exactly is a “streamer”?

Transferring data — or packets — from one point to another is the basis on which the internet is built on. In order to transfer said data, we use an Internet Protocol: a set of “rules” through which we move information from point A to point B. There are two widely used internet protocols which are in use today, each with their respective pros and cons: TCP (Transmission Control Protocol) and UDP (User Datagram Protocol).

TCP offers high reliability of data transfer. It acknowledges every single packet that goes through, and signals the sender to re-transmit data that didn’t come through. Additionally, packets are also sequenced to ensure that they’re transferred in the correct order. The “cost” of TCP’s great reliability is its reduced speed: the extra time required to check for errors renders it unusable for services that require real-time streaming.

UDP forgoes all error checks and simply delivers the packets as is. As such, it allows transferring packets quickly, albeit with no insurance that the designated end receives them. This protocol is the go-to standard for delay sensitive applications, such as real time decision making or streaming.

A “streamer core”, as we so playfully dub the tech, is a program that transfers media in real-time over a network using one of the aforementioned protocols. Beyond the protocol used, there are many different nooks and crannies that need to be adjusted. In order to offer the best possible experience, a “streamer core” has to be tailor made to the service that’s being provided.

It should come as no surprise, then, that Simplay’s ambitious service necessitated a custom-built streaming technology: A streamer core that deliver audio, video, and input (controls) all in sync and in minimal latency. With PHOTON, we were able to achieve just that — but that wasn’t always the case. Sometimes you have to start small before you take the big leap.

Humble Beginnings

Although UDP was unequivocally the protocol necessary to make game streaming happen, it was not without its issues. The lack of insurance as to whether the packets arrived at their target destination presented an obvious challenge: missing even the occasional packet means missing parts of a frame. That means noticeable pixelization, which results in a very poor gaming experience.

Clearly this had to be fixed, and we therefore turned to ENet: an open source project which sought to remedy UDP’s shortcomings by incorporating necessary elements from TCP. The most important feature the program offered was its sequencing capability: it assigned each packet with a incrementally increasing sequence number, which were then sent in a specific order. Incorporating serialization significantly reduced the lost-frames count, resulting in a greatly improved streaming experience.

Although ENet’s application was successful as a proof of concept, it was not without its shortcomings. For starters, the client would instantly and abruptly crash at the very whiff of network instability. It failed to recognize the packets sent as that of streaming data. Pixelization plagued the stream as too many packets still managed to get lost in the void. We did our best to push this open source library to the max, but we eventually realized it wasn’t going to cut it as far as our vision was concerned.

With that conclusion in mind, we began working on the next iteration of our streaming technology from the ground up. The result is the culmination of knowledge and experience we’ve acquired from studying ENet and various projects.

Enter Photon

Although small in number, our dedicated crew had managed to improve upon nearly every single facet of our previously used streamer. Having the project built from the ground up has allowed us to truly engage with the unique challenges that long-range game streaming presented. We were able to completely replace ENet, as well as almost any other open-source library we previously used.

But we didn’t stop there. A smooth stream, one that delivers consistent frame-rates, is the cornerstone of any good Cloud Gaming service. Lack thereof could prove to be a death sentence, especially as far high-octane gaming is concerned. This is why we included innovative frame-loss compensation algorithms: PHOTON takes into account a variety of parameters which ensures your game runs uninterrupted, even under less-than-ideal network conditions.

We also buffed up the streamer’s resilience so it could better handle unstable network connections. PHOTON is capable of handling much more abuse from the network, allowing it to bend and adjust to fluctuating network speeds rather than outright breaking down.

Overall, sweeping optimizations were made across the board, reducing the overhead latency to a mere 4–7 ms (a 10–15% improvement). Check out PHOTON’s streaming capabilities below:

The future looks bright

Our goal is to make game streaming truly indistinguishable from playing locally. With PHOTON, we’ve made a huge leap in that direction.

So what’s next? Well, we still have some way to go before we reach that “can’t tell the difference” level of quality. Our roadmap is full with tasks that will further improve the streaming quality: adding a direct peer-to-peer connection between client and host, incorporating innovative algorithms to reduce bandwidth and much more. Our work is far from over, and the future looks bright!