More Worlds, More Power: The Road to Greater Capacity

If you've been delving deep, roaming the lands, or standing in a crowded GE recently, you?ve probably noticed that Old School is more alive than ever. The Platinum Age is real, and we couldn?t be more thrilled.

With so many new and returning players joining the fun, demand for game worlds has skyrocketed. We're working hard behind the scenes to meet that demand, and today we want to update you again on what?s been done, what?s coming next, and why this isn?t just a simple case of "add more servers lol".

We?ve recently spun up a whole bunch of new worlds (an additional 43 in the last three weeks) across the US, UK, Germany, and Australia, with more on the way! But here?s the catch: Throwing new worlds online isn?t quite as straightforward as it might seem.

Behind each game world is a complex infrastructure, and if we just keep stacking on more without addressing the underlying systems, we risk creating performance issues, lag, and instability across the board. That?s obviously not something we?re okay with! So, what are we doing about it?

• We?re continuing to roll out new worlds as player demand continues to rise. These will be added in a staggered fashion to help us monitor performance and avoid overloading our current capacity. Next week we're aiming to add an additional 20 US worlds!
• We?re upgrading our infrastructure to remove limits and give more breathing room to each world. Work is ongoing and progress is accelerating towards being complete!
• We?re launching cloud-hosted worlds, starting with US East Coast, to offer better scaling and future-proof our server architecture. We're aiming to have this done within the next month.

What This Means for Regional Servers

Once our cloud-based hosting is running smoothly, it?ll unlock something players have been asking about for a long time: regional servers in new territories.

Rather than relying on physical hardware in every corner of the world (front-loaded capital expense, logistically tricky), this shift lets us scale into new regions without setting up shop from scratch. That means we?re one step closer to supporting players in regions like South America, Southeast Asia, or beyond.

We?re not quite there yet, and each new location will still require some bespoke work, but we hear you, and we?re building towards it!

If all you're interested in is a quick TLDR; then this is where you can stop reading. If you're interested in learning more about our server infrastructure, keep on scrolling...

Spudworks: What Powers Your Game World?

At this point we turn to our wonderful Spudworks from the infrastructure team who can peel back the layers behind our worlds much better than anyone else!

If you're unable to see the image above, click here!

With great player counts come great server responsibilities, and the hamsters are working overtime. Let's quickly explore how OSRS Worlds are currently hosted, what we are working on in the immediate term to provide more capacity, and steps we are taking to make our World hosting capacity more elastic going forward.

Being a Spudworks post, there?ll be a bit of tech-jargon to deal with and - as we in the Infrastructure Team are such a utilitarian bunch - graphs and diagrams lifted directly from internal sources, but we'll do our best to explain as we go. Let?s get started!

Hosting an OSRS World

We'll start with how Jagex hosts OSRS Worlds today. There are often comments speculating about this so let's take a peek.

Game worlds are run on virtual machines (multiple software-defined computers within a single physical computer, known as a Hypervisor). These virtual machines are hosted in regional commercial Data Centres to reduce latency between the Worlds and you lovely lot.

Public Cloud, in our case Amazon Web Services, is deployed at Jagex, but right now its use is confined to our core infrastructure, which provides global services such as the Jagex Accounts system, Player Saves, and data analytics.

If you're unable to see the image above, click here!

Jagex-ey Royals

For those interested, here's a very quick look at our hypervisors:

• We employ 4Ghz Intel CPUs. These are reasonably high-frequency for server grade CPUs, where there are trade-offs between single core performance, overall core density, and thermal output.
• We run VMWare vSphere with Linux Virtual Machines but may move away from VMWare in future.

Providing more Worlds

More players, more worlds! We hear you! As much as we?d love to simply double the World count, things get... complicated.

While we have some head room in our Edge Tier, adding a large number of new Worlds will increase the ratio of Virtual CPUs to Physical CPUs. Allocating more Virtual CPUs than you have Physical CPUs is called overcommitting and, while common in virtualised environments, pushing this ratio too high will impact World performance as many busy Virtual Machines compete for execution on a finite pool of physical CPUs.

As some of you have pointed out, the game can start to feel a bit clunkier when Worlds start to fill up! Contention for CPU time, as described above, can be a contributing factor (one we aim to minimise) but there are software factors too.

To shine a light on the software side of tick performance, we're handing the reins over to the OSRS Engineering team to share some insight:

Inside a Game World

Referring to our servers as "game worlds" is an apt description, as each server is literally simulating the entire world of Gielinor. That's a big world, with tens-of-thousands of NPCs, and up to 2,000 players! Those NPCs need to wander around, and those players need to access their banks, and the trees they chopped down need to respawn! And, each tick, we need to send some messages back everyone's devices, to let them know what changed.

A game world has a lot of work to do every tick, and one of our jobs is to make sure it all happens as quickly as possible. Many of you will know that a game tick lasts 0.6 seconds, but for the game to feel responsive, we aim for the actual processing time to be much lower. We also aim for that processing time to be consistent each tick, else the game's 100bpm metronome feels off.

It should be fairly obvious that the more players that are on a game world, the more scripts we have to execute, and the more work our core systems need to do.

There's also 20 years' worth of different things that a given player could be doing, and the amount of processing time required for each varies wildly! Right now, while our game worlds are performing reasonably well under load, it's a bit worse than we'd like. So we need to be keeping a careful eye on server performance.

Thankfully, we can monitor this in a lot of detail to find out where the server is spending most of its time. We keep a list of the top-200 most expensive gameplay scripts, and constantly chip away at them to maintain the health of the game. When Varlamore launched, an unexpected challenger scuttled into first place: the Gemstone Crab! This pointed us straight to an inefficiency in its code, which we patched up quickly.

Beyond scripts, we are always on the lookout for optimisations throughout the core engine. The script runtime itself more than doubled in speed from an optimisation pass in 2022, and more recently the system responsible for transmitting area updates to your client has had a huge overhaul in preparation for Sailing, bringing with it faster algorithms and improvements to CPU cache efficiency, that save several milliseconds off of each tick!

Now, we'll hand back to the infrastructure team to share our next steps!

Immediate steps

We'll continue to add more worlds to help meet the increased demand. We plan to roll these out in phases, rather than all at once, to let us monitor their performance impact.

This week, we've added 15 new worlds and converted 9 F2P worlds into Members worlds; next week, we are planning to add another 20 in the US!

Next step: Cloud-based burst capacity

Since before the recent growth in player numbers, we have been beavering away on migrating our Core to AWS and making AWS more interoperable with our existing on-premise infrastructure.

You may have noticed on the previous diagram that EU-1's Backbone lines are already orange, representing the availability of AWS Direct Connect (DX) circuits (dedicated networking between Jagex and AWS). Adding these DX circuits everywhere will allow our regional Edge Tier locations to burst sideways into AWS when extra capacity is needed.

This work will also enable us to support new, entirely AWS based regions where appropriate.

We?re not quite there yet, but the wheels (and hamsters) are in motion.

If you're unable to see the image above, click here!

We know you care deeply about how the game plays, and we?re committed to making sure your experience stays stable, smooth, and fun!

Thank you once again for being part of this incredible journey. And welcome to all our new adventurers ? you picked the perfect time to join us.

Got thoughts, questions, or potato-based theories of your own? You can discuss this update on the 2007Scape subreddit, the Steam forums, or the community-led OSRS Discord.

Mods Abe, Abyss, Acorn, Arcane, Archie, Argo, Ash, Ayiza, BigRig, Blossom, Boko, Bruno, Chilly, Criminal, Crystal, Curse, Daizong, Donkey, Ed, Elena, Enigma, Entropi, Errol, Freddie, Fuzz, Gecko, Gengis, Gizmo, Goblin, Grace, Grub, Halo, Harold, Hend, Hornet, Husky, Ivory, Jalo, Jamesy, Jerv, Keyser, Kieren, Kirby, Kurotou, Leebleh, Lenny, Light, Liron, Lykos, Mack, Manked, Markos, Maylea, Mobius, Moogle, Morty, Moz, Necro, Nin, Nixon, Nox, Nylu, Ori, Other, Philomel, Pork, Pumpkin, Puppi, Rach, Redfield, Rice, Roq, Roy, Ry, Saiyan, Sarnie, Schmidt, Shogun, Shroom, Sigma, Skylark, Smithy, Sova, Squid, Starry, Suharun, Surma, Sween, Tide, Titania, Titus, TJ, Tomb, Tsourorf, Tyran, Veda, Vegard, West, Wolfy & Yume

The Old School Team.

Mods Kraken, Vxp, Bash, Maniac, Vallcore, Haydon, Cky, Qwert, Drax, M0iqp, Ibex, Adad, & Roman ... and 🐹

The Infrastructure J-Mods.