Speeding up Unreal Editor by not opening 5500 files

In my last article I wrote about some tooltip optimization to reduce the start time of the Unreal Editor by 2-5 seconds. Turns out people do really care about their editor start time. So much that it did turn into my most read article to date. The need for speed is very real!

Taking a look at feature content packs

What would be better than to look for the next time saver? Surely there are still some seconds left to squeeze out of the code, right? However, before we dive deep into Superluminal traces, let’s take a look at a feature introduced with 5.0, the option to add a “Feature of Content Pack” to already existing projects:

This small menu entry offers a feature that is rarely used in ongoing projects but quite helpful in test projects or during prototyping, namely to add the content of Unreal’s Templates to a project after it has been created. This includes stuff like the basic shooter or top-down controls, which were only available during project creation in UE4.

“That’s all well and good, but what does this thing have to do with my editor’s startup time?” you might ask. Well, did you ever wonder how Unreal knows which templates and content packs you have installed? After all those packs are not required to exist, you can choose not to install them in the first place or can even add your own content packs in there, so there has to be a system to figure out what should appear in this list.
The way Unreal determines which content is available is quite simple, but also a bit scary – at least if you like your tools to work fast. Every time you start the editor, no matter if you ever planned on opening this specific dialogue, it will scan a short list of folders where content packs are stored. Then it will iterate over this list, looking for the manifest files (and .upack files) that describe the respective content pack. Which – in a default installation of 5.6 – means it needs to list more than 5500 files.

Even though Unreal is merely listing the files and is not reading their actual contents, the files have to be “opened” by the OS:

Opening 5500 files is of course a slow process. And this doesn’t even include the actual listing of all those files and needed string copies. In sum all those steps take (at least in my testing) between 1.5 (dev build) and 3.0 seconds (debug build).

How do we fix this?

My first instinct was to make the actual search faster. But before making any code faster, it’s useful to ask yourself if the code does even need to be run at all? Turns out that, in this instance, the answer is mostly no. After all, most times when starting the editor, users are not adding content pack to their project. So why do run all this during every time?

Similar to my last optimization I removed the call to the slow Refresh-function from the startup phase. Instead we refresh the list of content sources the first time any part of the editor tries to access it:

And ta-da: Already done. It’s not a perfect solution since we now have a short but noticeable UI freeze when opening the “Add content pack” dialogue, but forcing a few users through a small freeze is already much better than forcing all users through a longer startup phase.

You can find the pull request for this change here, feel free to give it a try.

(Note: To be able to see Unreal engine pull requests you need to be logged in to a GitHub account that is connected to your Epic Games account, see this page for details.)

Optimizing the Search

Originally, I wanted to end the article here, but I was not happy with simply calling the slow code less often. Three seconds to find some files seems really slow, surely the function itself could be written to be faster, right? Let’s take a look at the function that collects all the packs. Or rather a shortened version to make it easier to read in the context of this article, but I promise I didn’t remove anything important, you can check the according pull request for the full context.

This is basically a manual search. First, we create a list of all files in two given folders to then search manually through this list. As I mentioned earlier, iterating through and listing the files is the slowest part of the process. This is also confirmed by Superluminal: 83% of the time spent in this function is spent on those IterateDirectory-functions:

This seems quite wasteful, surely there has to be a way to already filter for filenames while going through the folder?

After a brief search, I found that there is indeed one. The IPlatformFile interface offers a similar function called “FindFilesRecursively”, which does exactly that. So let’s simplify this code.

Now let’s profile the new version to see if it can beat the 3 seconds of the old version:

A 6x speed boost, just by using the correct function, nice. I went ahead and created another pull request. Mission accomplished!

Could we still make this faster? Definitely, for example the screenshots of the contents packs are also loaded here (inside the FFeaturePackContentSource constructor), blocking the thread for around 0.1 seconds. Handling this part asynchronously would be great, but for now I’m quite happy with this implementation.
If you are too, consider following me and my blog, so you don’t miss the next article where I write about Unreal shenanigans and other Game dev topics. You can find me on BlueSky, Mastodon and even LinkedIn. Also consider sharing this article if you found it useful, or send me your ideas how to speed up Unreal even further. I might write another entry in what is starting to turn into a mini-series of articles on editor startup times 🙂