Netcode Series Part 2: Data Channels (Snapshots and RPCs)

The in-game network monitor widget is hard to read, but never lies.

Ok, so we have a client running a phone, a PC, a console, it doesn’t matter. What matters is that you’re talking with a dedicated server through a connectionless communication channel, and it’s time to make things move.

The focus of this article is identifying what data is core to the gameplay experience, what’s not, and the process of deciding how the information gets across.

When implementing any feature, you need to ask yourself the 2 golden questions:

Is the data latency important?— Is the data essential to game state?

Speed über alles

Printed on countless t-shirts, only funny to a few.

UDP is the devil you know, and the one you need to get things done fast. We won’t deep-dive into the protocol, but suffice it to say it’s used for the most demanding real-time multiplayer games and audio/video communications.

If you’re interested in knowing more about UDP and building a protocol on top of it, Glenn Fiedler has you covered once again.

If you don’t consider network congestion, UDP is faster than TCP and with a smaller footprint. To be that fast, you need to shed some weight:

• Connectionless communication. There’s no built-in handshake or packet ACK, which means you’re not really “connected”. If you know the port, you can start sending packets, no strings attached.
• No packet resend on loss. Your data may arrive, or it may not. Cross your fingers! 🤞 You can decide what’s important and resend it, and discard what is not.
• Unordered packets. If your data arrives, it may be too late or too… soon?

So, TCP is like a bumper to bumper highway. There’s no escape until all of the cars have reached destination. In UDP, you got your flying cars that may stray off the main path but are whizzing by each other.

It’s really important to understand what the game needs, and for which functionality. Not having to deal with packet congestion is great, but you need to build a reliability layer.

With ARMAJET, we use 2 channels to build all gameplay features on:

• Unreliable, Unordered channel. For core gameplay and non-essential data. The game state travels on top of this as well as some events.
• Reliable, Ordered channel. For latency-tolerant events and commands. Chat and RPC-style calls are done here.

Unreliable, Unordered: Core Gameplay Snapshots

All core gameplay is built on the unreliable, unordered channel. Before jumping onto what that means, it’s important to first define what Core Gameplay data is.

Core gameplay data is any data necessary to define the state of the game at any point in time.

The easiest way to think about it is, let’s say you connect to a game already in progress. You need to know who’s playing, the score, if any projectiles are flying by, etc.. Basically, the data required to build the scene to display.

Core gameplay answers a resounding Yes to both our golden questions:

Is the data latency important? Yes — Is the data essential to game state? Yes

All entities such as Game mode, Players, Projectiles, etc. make up the Gamestate. Entities can spawn, carry changing properties and despawn. Entities carry a limited amount of pre-determined properties called the Net State, which is the contract needed to run our bit-packing (yes, bits, not bytes!).

Net states are written out as protobuffer definition files, a format we created to quickly spell out variables and their serialization format. They’re then automatically parsed and exported into C# files with utility functions. This is what a paired down projectile net state looks like:

class ProjectileEntityNetState : PropEntityNetState
{
   var InstigatorID : ushort;
   var VelX : float<0.001>;
   var VelY : float<0.001>;
   var IsWallStuck : bool;
   var FireTime : double;
   var WeaponID : ushort;
}

These support inheritance, so the PropEntityNetState also carries Position and Angle properties. All the entity net states put together are the Gamestate.

Here’s where it gets interesting.

The Gamestate is saved on the server at every simulation tick. Tickrates in popular shooters are known to be as low as 20hz (Overwatch) and as high as 128hz (Valorant). Every tick stores the Gamestate into a circular buffer of Snapshots. Delta Snapshots are the key to effective data compression.

Delta snapshots are messages that are custom-tailored to each player, and they works like so:

• The server sends snapshots periodically to all clients connected. The first snapshot is the Gamestate in its entirety, including all entity net states (that’s data pertaining to game mode, player entity, projectile entity, etc.).
• Since we can’t rely on the protocol ordering packets for us, each snapshot has a sequence number. If the client receives a snapshot that has an older sequence number than the last received, it is discarded, as it contains old data we don’t need (like a previous player’s position).
• When the client receives a snapshot, it also acknowledges the receipt to the server by sending the sequence number.
• The server now knows what the client knows. It can generate custom-tailored snapshots based on the last acknowledged sequence number, sending the client only the data that has changed. This is what’s called a delta snapshot.
• If there is packet loss or jitter, there will be a delay in snapshot acknowledgement. This means the server will send increasingly larger snapshots, including more and more changed data that was never received by the client.

What’s important to note is that no matter which delta client snapshot is the latest, a full game state can be reconstructed, meaning core gameplay state is maintained and the simulation can continue running.

You can find out more about the technical implementation of Delta Snapshots on Fabien Sanglard’s deep-dive of Quake3’s source code here.

I should note that Snapshots aren’t suitable for all games. They’re great for non-deterministic physics-based games that require frequent synchronization on a limited amount of entities. If you have a fully deterministic game (meaning the same inputs will produce the exact same result each time) such as a fighting game, input rollback systems are a better, more efficient way to go, with no need for positional deltas. If you’re making an RTS with hundreds or thousands of entities, you’ll need a different network model altogether.

Unreliable, Unordered: Events

These are the events describing volatile data that don’t break the game state if they don’t reach the client on time… or at all.

Is the data latency important? Yes — Is the data essential to game state? No

The railgun hit effect and confirmation. Not a big deal if the event packet is lost.

In ARMAJET, when a player fires a projectile that hits an enemy, a few things happen on the client side:

• The projectile explodes.
• The player’s health is reduced.
• Damage numbers appear on top of the enemy.
• Triggers sounds and visual hit effects, using angle and hit offsets.
• If killed, a log shows the shooter’s name, the weapon icon, and the victim’s name.

The player’s health change, as well as the projectile explosion are all part of the entity net state property changes, informed by delta snapshots.

Projectile hits, however, are not entities. They don’t spawn, carry changing properties, or despawn. They are discrete events. Snapshots can tell the client that the enemy health was reduced by 30hp, but they can’t inform if it was caused by a single 30hp hit, or 3x10hp hits.

Events travel on the same unreliable channel as Snapshots, but are not repeated or stored. In the rare occasion when a client misses the packet describing the hit, the opponent’s health reduction will still reflect the hit(s) and the victim’s demise, reliably. The client can still reconstruct the damage hit from the snapshots, it just won’t know the exact projectile that caused it on the server.

It’s important for the developer to create a rendering engine that can tolerate missing data and delays, so when they do ultimately happen, the player isn’t faced with a buggy, glitchy-looking experience. Or even worse, completely different outcomes (Worms replays, anyone?).

Reliable, ordered

This reliability layer replicates the core features of TCP, without some of the overhead, or the need to use another protocol/port. We use this for any classic RPC-style commands.

Is the data latency important? No — Is the data essential to game state? No

Examples of this are:

• Chat
• Remote cheats
• Selecting Loadouts/Spawning
• Changing teams

Client and Server are guaranteed to get these in the right order, eventually.

Today we went over how gameplay data is split up into channels, and how to decide how the data needs to be transferred to maintain reliability and keep bandwidth use reasonable.

In the next article, we’ll discuss how player movement works in a server authoritative environment, and how much there is under the hood to keep players looking smooth.