Booting into Toolbox Containers

There are a lot of tangible benefits in using toolbox containers for development to the point that I don’t want to use anything else anymore. Even with a bunch of tricks at our disposal, there are still downsides. The containers are not complete sessions but rather try to integrate with the host session. If you’re working on something that is part of a session it might be possible to run a test suite and even more elaborate setups but running it fully integrated often becomes a problem.

If the host is a traditional mutable system it’s possible to just not use toolbox. If you’re on an immutable system they often offer some way to make it mutable temporarily using some kind of overlay at which point they behave mostly like the traditional mutable systems. The unfortunate side effect is that you don’t get the benefits of toolbox anymore.

It’s also possible to develop in the toolbox and on the host system at the same time, depending on what specifically you’re working on right now to get the benefits of both systems. The drawback is that the toolbox container and the host are different systems. You’re setting up, compiling, etc. everything twice and run your project in different environments. Also not ideal.

We can do better. Toolbox can, in theory, use arbitrary OCI images. In practice there are assumptions from toolbox on how an image looks and behaves. Fedora Silverblue, or rather rpm-ostree, can also, in theory, boot arbitrary OCI images but also comes with its assumptions.

It turns out that in practice the unofficial OCI image variant of Fedora Silverblue can be used as a toolbox image and the images of such containers can be booted into with rpm-ostree.

$ toolbox create -i quay.io/fedora-ostree-desktops/silverblue:39 my-silverblue-toolbox
$ toolbox enter my-silverblue-toolbox
⬢ # install dnf to make it behave like a usual toolbox container
⬢ sudo rpm-ostree install -y dnf
⬢ sudo dnf update -y
⬢ # Let's install strace and gdb. Do whatever you want with your container! 
⬢ sudo dnf install -y strace gdb
⬢ exit
$ # some magic to convert the running container into something rpm-ostree understands
$ # there are probably ways to do this with less copying (tell me if you know)
$ podman commit my-silverblue-toolbox my-silverblue-toolbox-image
$ sudo mkdir -p /var/lib/my-silverblue-toolbox-image
$ podman save --format=oci-archive "my-silverblue-toolbox-image" | sudo tar -x -C "/var/lib/my-silverblue-toolbox-image"
$ sudo rpm-ostree rebase "ostree-unverified-image:oci:/var/lib/my-silverblue-toolbox-image"
$ # boot into our toolbox
$ sudo systemctl reboot -i

One toolbox container to develop in and one reboot to test the changes in a full session on real hardware. This is all unsupported and might break in interesting ways but it shows the power of OCI based operating systems and toolbox.