Nix: Connecting to the Sandbox

3 weeks ago 1

Wait, the sandbox is basically a container? Always has been

I’ve been fighting with an annoying issue for a while now. There’s a Storybook build which works in my local devshell but fails when building in a derivation. More annoyingly, the devshell is derived from the derivation, so the impedance mismatch is quite low ™.

Now, in this particular case, the toolchain wasn’t being of much help. Vite was telling me there was an unhandled exception somewhere, but for neither love nor money could I get it to tell me where!

Having admitted defeat, I was in the Clan Jitsi channel handing it over to hgl, a real frontend developer, when Qubasa piped up and said:

You know how to connect to the sandbox, right?

Now, I’ve been using Nix for a few years. I’m no Mic92, but I’m no noob either. I like to think I can use the repl better than most, and I’m not too shabby when it comes to writing derivations.

The conversation continued…

[Me] You mean using `nix develop` to drop into the same environment within your current shell right? [Qubasa] No, I mean forcing your build to pause and connecting to the sandbox directly. [Me] You can do that?

It turns out, you can!

Patrick from SpongeBob SquarePants with his head exploding

It’s basically a container

Without getting into the details, at a high level you could say that the sandbox is basically a container. That is to say, it uses a lot of the same underlying technologies as containers. Which means you can use a lot of the same tools as you would with containers.

Now, in cooking the phrase “here’s something I made earlier” gets thrown around a lot. Within Nix, that tends to be “here’s something Mic92 made earlier”. So as you may have guessed, Mic92 has written something which can help us out.

Cntr is a nifty little tool for container debugging. And since we’ve learned that the sandbox is basically a container, we can use it to connect to the sandbox directly.

Let’s use hello as an example.

# test.nix rec { # get ourselves an instance if nixpkgs pkgs = import <nixpkgs> {}; # force a sleep in the build so we can find the process and connect to it hello-sleep = pkgs.hello.overrideAttrs { preBuild = '' # sleep for 1 hour to force the sandbox to pause sleep 3600 ''; }; }

If you run nix build -f test.nix hello-sleep, you’ll see that the build pauses. In a separate terminal, run ps -aux | grep sleep and look for the nixbld* user running our sleep command:

❯ ps aux | grep sleep brian 853708 0.6 0.0 726200 91932 pts/5 SNl+ 15:20 0:00 nix build -f test.nix hello-sleep nixbld1 857382 0.0 0.0 4904 2624 ? S 15:20 0:00 sleep 3600 brian 857418 0.0 0.0 230776 2936 pts/8 SN+ 15:21 0:00 grep sleep

With the process ID, we can now use cntr to connect to it:

❯ sudo cntr attach 857382 nixbld@saturn:/var/lib/cntr/ > ls -alr total 0 drwxr-xr-x 1 nobody nogroup 6 Oct 15 14:23 var drwxrwxrwt 1 nobody nogroup 44 Oct 15 14:23 tmp -xr-xr-x 719 nobody nogroup 0 Oct 15 14:20 proc wxr-xr-x 1 nobody nogroup 10 Oct 15 14:20 nix -xr-xr-x 1 nobody nogroup 32 Oct 15 14:20 etc wxr-xr-x 1 nobody nogroup 120 Oct 15 14:20 dev wx------ 1 nixbld nixbld 40 Oct 15 14:20 build wxr-xr-x 1 nobody nogroup 4 Oct 15 14:20 bin wxr-xr-x 1 nobody nogroup 780 Oct 15 14:00 .. wxr-x--- 1 nobody nixbld 54 Oct 15 14:23 . nixbld@saturn:/var/lib/cntr/ >

Et, voilà! 🥳 🎉

There’s a better way

You might be thinking it’s a bit hacky dropping in a sleep command, and you’d be right. As it turns out, there’s a breakpoint hook which can be used to pause the build in the event of an error.

# test.nix rec { # get ourselves an instance if nixpkgs pkgs = import <nixpkgs> { }; hello-breakpoint-hook = pkgs.hello.overrideAttrs ( _prev: _final: { preBuild = '' # force an exit with error exit 1 ''; nativeBuildInputs = [ # add the breakpoint hook pkgs.breakpointHookCntr ]; } ); }

When you run nix build -f test.nix hello-breakpoint-hook, you’ll see that the build will pause and give you a handy command to run to connect to the sandbox:

❯ nix build -f test.nix hello-breakpoint-hook [1/0/1 built, 1 copied (0.0/0.0 MiB), 0.0 MiB DL] building hello-2.12.2 (buildPhase): cntr attach -t command cntr-/nix/store/hvfi8i5x8ab3wqlm4g42vhypfidryvg5-hello-2.12.2

As the output suggests, this is using cntr under the hood just like we were above.

Whilst you may consider breakpointHookCntr the OG breakpoint hook, there is a more recent and canonical one, breakpointHook, which ensures some env setup is performed correctly when you connect and gives you a fully qualified path to an attach script so you don’t have to ensure cntr is in your PATH:

Similary to breakpointHookCntr, you can run nix build -f test.nix hello-breakpoint-hook and you’ll see a command to connect to the sandbox:

❯ nix build -f test.nix hello-breakpoint-hook [1/0/1 built] building hello-2.12.2 (buildPhase): sudo /nix/store/r6bgqs87nyg7hrwxgi1gi0h48lwm5b3a-attach/bin/attach 243181