Easel's State-of-the-Art Rollback Netcode Implementation

The speed of light is not as fast as we would like.

When players are far away from each other, there is a delay between when the player sends an action and when another receives it. This delay is called latency, and it can make games feel unresponsive. Imagine pressing the jump button and your character not jumping immediately. Easel uses rollback netcode to make a game feel responsive even when they are far apart.

This page explains what rollback code is, and what makes Easel's rollback netcode implementation unique.

What is rollback netcode?​

Rollback netcode is a technique for hiding latency in multiplayer games. Every client simulates the entire game locally using all the inputs they have received up to that point in time. Because players are distant from each other, sometimes inputs arrive late, after the game has already simulated past that point in time. If that happens, the game rolls back to the point in time where the missing input occurred, and then re-simulates the game from that point forward using the complete set of inputs. In this way, multiplayer games become eventually consistent between all players, even when there is high latency between them.

Why does rollback netcode work?​

Generally players only provide input a few times a second, while the game is simulated 60 ticks per second, and so most frames can be predicted accurately without needing to wait for all inputs to arrive. That is why rollback netcode is able to hide latency so well in practice.

Easel's state-of-the-art rollback netcode implementation​

Easel has one of the most advanced rollback netcode implementations of any game engine. This section details the unique features of Easel's rollback netcode.

Incremental snapshotting and rollback​

Many rollback netcode implementations must snapshot the entire game state at every tick. Sometimes this takes longer than executing the game simulation itself. In Easel, only the changes are snapshotted and rolled back, not the entire game state, which improves performance dramatically. Because multiplayer is built into the Easel programming languages, the change detection happens transparently and reliably without any effort on your part.

True peer-to-peer, no host peer​

Easel uses peer-to-peer networking because this means every packet can take the shortest, most direct route between players, minimizing latency.

A common design of peer-to-peer games is for one of the player's computers to act as the host. This gives the host an unfair zero-latency advantage, while doubling the latency for all other players as they must wait for a full roundtrip to the host and back.

In Easel, there is no host player. Inputs are sent to all other players equally, as quickly as possible, and every client forms updates its own view of the game whenever the inputs arrive. Latency is distributed fairly to all players with no one getting an unfair advantage, and no one suffering doubled latency.

This is only possible because rollback netcode enables clients to start forming a picture of the world even before they have all inputs.

Perfectly-located servers​

In traditional multiplayer games, the servers are in a fixed location, and they are never quite perfectly placed to the current set of players that are online right now. Easel's peer-to-peer architecture means that the virtual "server" is effectively located at the exact midpoint between all current players, giving them the best possible latency for the players right now.

Because the "server" is virtual, it can sometimes be placed in geographically impossible locations, like the middle of the ocean, which is how Easel is able to achieve lower latency than traditional multiplayer architectures could ever hope to achieve.

Flexible regions​

Traditional multiplayer games have fixed servers in fixed locations, and so players must select a server region to play in. This subdivides the player base and makes it harder for them to find each other. As Easel games are peer-to-peer, it can match players more flexibly. Easel uses geolocation to determine which players are close enough to each other to play together, regardless of if they are on different continents. This means it can better serve multiplayer games with smaller player bases.

See Regions to learn more about how Easel matches players together.

Relayed through Cloudflare​

Peers connect to each other through their nearest Cloudflare servers, which protects their IP address, and also ensures that players behind NATs and firewalls can still connect to each other. Cloudflare has an international network of 400+ data centers, with most people on Earth being within 10ms of a Cloudflare data center, and so this gives excellent performance for most players while protecting their privacy.

Fan out by Cloudflare, not client: Normally in a peer-to-peer architecture, if a player must connect to 10 other peers, it must send 10 copies of the same message, which can cause congestion on the player's network connection and affect their latency. In Easel, the message is sent just once to Cloudflare, and then Cloudflare's server fans it out to the other peers. The Cloudflare servers are connected to each other through high-bandwidth links so they have no trouble handling the load.

Fair latency distribution​

Sometimes not all latency can be eliminated by rollback netcode. In these cases, Easel assigns the latency so that each player experiences the latency they contribute to the game. If one player is far from the others, they themselves experience their own high latency and the other players are unaffected.

This is the fairest way to handle latency in multiplayer games. Other rollback netcode implementations are only able to assign latency to all players equally.

Automatic rubberbanding​

When a rollback occurs, the predicted game state may differ from the actual game state. Easel uses rubberbanding to smooth out prediction errors, making them less jarring to players. This happens automatically and most game developers won't even know it is happening!

Server-authoritative​

Even though messages are sent peer-to-peer, the Easel server still has ultimate authority over the input sequence. If there is any disagreement between peers, the server's version of the truth wins, meaning it is impossible to cheat.

Lag spike protection: One of the biggest problems with peer-to-peer is one player can introduce lag for everyone else, and if there is a lag spike, it is difficult to confirm which side is at fault. As Easel uses a server-authoritative peer-to-peer model, the server is able to detect when one particular player is causing lag for everyone else, so it is able to mitigate lag spikes from individual players, ensuring a smooth experience for everyone.

Continuous time synchronization​

When packets arrives late, Easel must decide whether the difference is due to either latency or clock differences, and so good time synchronization is essential for rollback netcode to work well.

A common way to do time synchronization is to measure the round-trip time of packets. The packets need to return as fast as possible to get an accurate measurement, which means the network must be uncongested. This is not practical - in games we need to be downloading assets, and sending and receiving inputs all the time. Players do not want to wait 5 seconds for the network to do nothing except synchronize clocks.

Easel uses an innovative in-band time synchronization algorithm which piggybacks on the existing game inputs messages themselves that need to be sent anyway. This makes it both reliable and efficient.

The hard work is done for you​

Implementing rollback netcode correctly and performantly is fiendishly difficult, but we have done all the hard work for you.

Guaranteed determinism​

For rollback netcode to work, the game must be deterministic, which means it must always produce the same output given the same input. There are many ways that non-determinism can creep into a game - floating point calculations, random number generation, inconsistent trigonometry functions, differences in order of execution, and so on. We have painstakingly made sure that anything written in the Easel programming language is always deterministic. You never need to worry about accidentally write non-deterministic code in Easel, and so you can never cause desynchronization bugs yourself.

Snapshotting executing code​

Easel games are sometimes made up of thousands behaviors, all executing concurrently. Easel incrementally snapshots the execution state of all behaviors, and restores them correctly during rollback. We have never seen a rollback netcode implementation that can snapshot code mid-execution.

A programming language built for multiplayer​

Easel was specifically designed for multiplayer games. We chose not to use embed Lua, JavaScript, or any other existing programming language because they did not go far enough to achieve:

• The precise level of deterministic execution required for rollback netcode
• The ability to snapshot and restore execution state of coroutines
• Our vision for how to make a domain-specific game programming language that is powerful enough to be interesting, yet simple enough to be accessible to everyone

No understanding required​

Easel multiplayer implementation is full of technological advancements, but ultimately, you don't need to understand any of them to make multiplayer games with Easel. You can just code as if all your players are in one shared world, like a singleplayer game, and Easel takes care of the rest. See Multiplayer to learn how to make multiplayer games with Easel,