First experiences with Nix shell

Migrate to Nix

The premise of Nix shell is that it makes available a fixed version of tools. This solves a common problem: how to make sure that all developers use the correct NodeJS, Ruby, Java, and they all have things installed, such as graphviz, gimp, and so on?

Before Nix, my go-to solution was Docker. In a Dockerfile I can control everything: what base system to use, what configuration to apply, what packages to install. Docker is great for completely reproducible environments.

But doing development in Docker is full of frustrations as well: since it's a completely separate environment it's hard to attach a debugger to a process inside and hard to move files in and out of the container. For example: a generated Epub failed validation, how do I figure out why? Normally, I could just write it into a file and observe it with tools, but with Docker it's not easy to extract that from the container. I wrote an article about some of the tips I came up with to solve these problems.

Related

Tips to debug NodeJS inside Docker

How to efficiently develop Dockerized NodeJS apps

Also, users inside Docker containers is a mess. Writing files to the host messes up with permissions ending up with directories owned by root that I need sudo to delete, and even inside the container some tools don't like to be run as root.

So, Docker remains my go-to choice when I need a reproducible environment, such as to run in a CI, but I needed something more ergonomic for development. And this is where Nix shell is helpful.

Using Nix shell

What I like about Nix is that I have access to all the tools that are available in my local computer and it just adds things on top. I can make sure that the correct version of graphviz, JS, Java, etc. are all installed, but I also can drop into my familiar NeoVim setup, access the full filesystem and network.

A shell.nix:

let
  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-23.11";
  pkgs = import nixpkgs { config = {}; overlays = []; };
in

pkgs.mkShellNoCC {
  packages = with pkgs; [
    jdk
		graphviz
		nodejs_21
		librsvg
  ];
}

Then use nix-shell to get a shell that has all the tools listed above available.

$ nix-shell

[nix-shell:~/workspace/awm/blog]$ node --version
v21.6.2

[nix-shell:~/workspace/awm/blog]$ dot --version
dot - graphviz version 9.0.0 (0)

[nix-shell:~/workspace/awm/blog]$ rsvg-convert --version
rsvg-convert version 2.57.0

It's also easy to use some Makefiles that wrap this:

.PHONY: clean clean_output

node_modules: package-lock.json
	nix-shell --pure --run "npm ci"

_site: clean_output node_modules
	nix-shell --pure --run "npm run eleventy --quiet"

clean_output:
	find _site -mindepth 1 -delete || true

clean:
	rm -rf _site
	rm -rf node_modules

Nix in Docker

But Nix is not available in GitHub actions, so I can't just use the same tools in CI as locally. So Docker is still needed.

Nix also comes as an operating system: NixOS, so I naturally started with that:

FROM nixos/nix:2.20.1

This revealed a couple of shortcomings in my shell.nix: since nothing comes preinstalled to the OS, I got a couple of errors first. Turns out some of the dependencies I use need gcc and python3, and the OS needs the cacert to have the certificates necessary for HTTPS.

let
  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-23.11";
  pkgs = import nixpkgs { config = {}; overlays = []; };
in

pkgs.mkShellNoCC {
  packages = with pkgs; [
    jdk
		graphviz
		nodejs_21
		gnumake
		cacert
		python3
		gcc
		librsvg
  ];
}

This was not an issue locally as my host system has all these. But starting from scratch I needed to explicitly define everything. This also reveals some weaknesses: besides using something like Docker to have a clean environment, how do I know that I defined everything? Maybe it works because it picks up a dependency from my own system and the next developer's build is still broken.

Puppeteer in Nix

Getting started was easy enough, but then I started to see some very strange errors:

789.5 Error: spawn /root/.cache/puppeteer/chrome/linux-121.0.6167.85/chrome-linux64/chrome ENOENT

Maybe puppeteer did not install Chrome properly?

Well, let's have a shell and see:

$ docker run -it sha256:530abb70748a0de730b92da5a39167983e4a5d02da29b836f47c334acdbc7007

sh-5.2# cd /root/.cache/puppeteer/chrome/linux-121.0.6167.85/chrome-linux64/

sh-5.2# ls -l
total 266560
...
-rwxr-xr-x 1 root root 237979160 Feb 16 11:58 chrome
...

OK, the file is definitely there. So, why does it say that it's not found?

After some searching, I found this article: https://blog.thalheim.io/2022/12/31/nix-ld-a-clean-solution-for-issues-with-pre-compiled-executables-on-nixos/ that describes the exact issue I'm seeing here: it's not that chrome is not found but its dependency. This is because the executable was built with the assumption that it will run on a Linux but now I'm trying to run it on NixOS.

So, what's the solution?

The article links to this tool: https://github.com/Mic92/nix-ld. It patches the executables to work on NixOS.

This is definitely an approach, but I decided to go the other way: start with Linux and add Nix on top of that. This immediately revealed a problem that it's not as easy as apt-get install -y nix. Fortunately, there is a project that makes it almost as simple: https://github.com/DeterminateSystems/nix-installer.

The Dockerfile with Nix available under Ubuntu:

FROM ubuntu:22.04

RUN apt-get update \
    && apt-get install -y curl

RUN curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --no-confirm --init none
ENV PATH="${PATH}:/nix/var/nix/profiles/default/bin"

RUN nix-channel --update

...

Fonts

Next problem I noticed: missing fonts. Ah, there is an article exactly about that: https://nixos.wiki/wiki/Fonts.

After some trial and error (mostly error), I realized that it applies only to NixOS and not the Nix shell. Fonts can be installed as regular packages (such as noto-fonts), but they won't be picked up. I'm sure that with some Bash magic it's possible to make them available to the system, maybe the approach detailed in this article, but I went the easier direction here: just assume that fonts are installed on the host.

Inside Docker, install the fonts:

FROM ubuntu:22.04

# ...

RUN apt-get update && apt-get install -y \
	fonts-noto fonts-ubuntu fonts-freefont-ttf fonts-liberation fonts-liberation2 fonts-opensymbol \
&& rm -rf /var/lib/apt/lists/*

This is not the ideal solution: while it makes sure that when running Docker all the required fonts are installed, between developer environments they can still be missing. This results in depending on the host system some fonts are available while some are not.

Conclusion

Overall, I'm happy with this setup. Nix shell solves a huge pain point as I can be sure that the required dependencies are always installed but does so without the burden of Docker's separated environment. It feels lightweight and keeps all my existing tools in place.

Linux executables and especially fonts are still a problem though but something I can live with for now. Things not rendering pixel-perfect is at the moment just a visual distraction during development, but I can see possible use-cases where it can be a problem.