Feature Preview: HDR and Lumen Global Illumination Raytracing

Welcome to a new little blog series we're doing in the run up to Early Access. We currently don't have time to write huge technical dev logs like we did in the past, but there are still a lot of new features to talk about! So, we decided to instead post a bunch of little blogs whenever something is ready without losing ourselves too much in the details.

Today we'll talk about a few optional graphics enhancements you can use if "Ultra" just isn't good enough for you and you would really like to use your GPU for heating your room in the winter.

Introduction

Before we start, you can watch this video demonstrating what it looks like with all the features enabled. We had to wait over a day for YouTube to process the video and actually make the 4k60 HDR option available. Isn't that crazy?👇

HDR Output, Episode 2

If you remember our post from a year ago, we actually talked about this before.

The only problem was that since that previous post, we had upgraded to Unreal Engine 5.5 just before the demo, and naturally Epic had managed to somehow break the HDR in the new engine version. This is what were greeted with after the update when a user enabled HDR and opened any UI:

Some serious skill issues here.

Luckily, this turned out to be easily fixed after some investigation in RenderDoc. They had merely forgotten to clear a render target before using it, causing all this garbage data to be displayed on screen.

There was also a second issue where having the avatar editor on screen turned the background black. That one was more involved, but is also fixed now - to render the avatar editor we now first blur the background in HDR, render most of the UI, render the avatar background on top of the blurred HDR render target and cut a hole into the UI where it should show, and finally composite everything back together into the final output.

These images were captured in 4k HDR using Xbox Game Bar on Windows 11, which writes a JPEG-XR file, and then converted to AVIF using this tool I found, so they can display in the browser. On my computer, the version in the browser looks exactly the same as the game, but I have no idea if that works for you too.

The avatar editor now has HDR. Whoa.

HDR test image 1.

HDR test image 2.

HDR test image 3.

HDR test image 4.

HDR test image 5.

HDR test image 6.

Some troubleshooting if the images don't display correctly:

• If you don't have a HDR monitor, you can't view them.
• If you have HDR disabled in Windows, enable it.
• If you are using Edge or Firefox or something else, use Chrome.
• If you are using Chrome but the images are too dark, change the HDR brightness balance setting in Windows, or change the dynamic tone mapping setting on your display's menu.
• If you are not using Windows, I can't help you.

Lumen & Hardware Ray Tracing

Remember how we implemented support for hardware ray tracing bricks a long time ago? That code had gotten a little scuffed up over the years and was no longer functioning in the demo, but I just fixed it up. Hardware ray tracing is now back for Early Access, it's also quite a bit faster since back then (but still slow...).

In the mean time, Epic had made lots of progress on Lumen. Since we are now on Unreal Engine 5.5, I thought I'd give it another try. We still don't have distance fields to be able to use software ray tracing, but what if we could make the hardware ray tracing path work?

Naturally the first attempt did not work at all. We weren't getting any information from HWRT to Lumen whatsoever:

Not even the wheels and character are working correctly, which only use basic engine meshes.

Since I had never really worked with Lumen before, I opened up an Unreal Engine example project to see how this view should look, and theirs is... quite different, wouldn't you say?

It's not black.

Turns out most of that was just because I had purposefully broken the lumen scene some time ago while making Proto Lumen. Let's revert that change so it only applies when we are actually running in Proto Lumen mode...

Our ground has started to show up. Wheels are still getting culled, but let's worry about that later.

The bigger problem is that our bricks are still broken.

All of the bricks are culled despite taking up all the screen. Why?

It turns out Lumen requires something called a Surface Cache to work properly. Basically, we have to cover our meshes with planes onto which it will render indirect lighting. Of course the code Epic made to do this for normal meshes only works in the editor and is extremely slow as it is intended for offline processing.

How do we do this? We could just write a more optimized version of their code that works at runtime. They are doing ray tracing against the meshes on the CPU side and it can take like half a second to process a single mesh this way. When we build clusters for a large build, we have thousands of meshes.

To do it at runtime, we'll do all of the following:

• We will use the embree4 library to trace the rays, making full use of its capability to trace 8 rays in parallel with AVX instructions.
• We will use our low detail mesh as input for each brick cluster.
• We will use very few rays, accepting a mediocre real-time result.

The full algorithm ends up as the following: it will take the low detail mesh we build for the brick cluster, convert this to an embree4 scene to be able to ray trace it on CPU, overlay a 32x32x32 voxel grid, then for each face of the grid (6) it will trace rays through the grid to hit the mesh, generating layers of surfels at the hit surfaces, trace additional rays to determine if some hits might be "inside" the mesh only and useless, then cluster those surfels to finally end up with the set of most significant surfaces to output as cards.

Our implementation of this is now so fast that it barely increases the time it takes to generate brick clusters.

Generated cards for a hollow brick structure.

We have full surface caching for all the bricks!

In real builds, it looks significantly more cluttered, but it works surprisingly well. We don't always cover every surface since the brick clusters may randomly include multiple hollow structures, but we get most of them which is enough for the whole system to work.

That's a lot of cards.

We've covered like 90% of the scene with at least one card, so I'm happy with the result.

Almost every build I've tried had ended up with a usable surface cache.

Let's get rid of those annoying debug views so we can appreciate the glory.

Yes, it does ray traced reflections too.

Almost feels like a real-time photo mode, rivaling the old path tracer we used to have. I've never seen a building game capable of looking this good before...

All the new options at a glance.

The memory usage and performance of this mode can be quite heavy. Epic has several performance optimizations in Unreal Engine 5.6, but we won't be able to get that in before release. If it doesn't currently run well on your computer yet, maybe it will once we upgrade to that.

Well, that's all for this preview. There are going to be many more before we reach Early Access. Stay tuned!