Introducing Nix

Declarative builds and deployments

codgician

2024-05-20

Problems

Software should still work when distributed among machines.
Not the reality. Following challenges:
- Environment issues
- Managability issues

Environment issues

Components have dependencies.
Dependencies need to be compatible.
Dependencies should be discoverable.
Components may depend on non-software artifacts (e.g. configurations).

Manageability issues

Uninstall / Upgrade: should not induce failures to another part of the system (e.g. DLL hell).
Administrator queries: file ownership? disk space consumption? source?
Rollbacks: able to undo effects of upgrades.
Variability: build / deployment configurations may differ.
… and they scale for a huge fleet of machines with different SKUs.

Industry solutions?

Idea #1. Global package management

Systematically manage packages (e.g. apt, yum, pacman, etc).
Each component provide a set of constraints:
- A is installed $\rightarrow$ B (>= 1.0) must be installed.
- A is installed $\rightarrow$ C should NOT be installed.
- Success deployment $\rightarrow$ pkg1, pkg2, … is installed.
Solve: success deployment.

B-SAT problem (NP-complete).

::: { .fragment .fade-in } Implement pseudo-solvers. :::

When you try to manage the global system,

you lose isolation.

Two components want different versions of the same dependency?
Two components providing files on the same location?

Upgrading is destructive, and not atomic.
Files are usually overwritten, making rollbacks non-trivial.

Idea #2. Go monolithic!

Environment issues?
- Resolving undeterministic dependency is hard.
- Why not bundle everything?
Managability issues?
- Isolation between bundles.
- Only load dependencies which are bundled inside.
Self-contained packaging: AppImage, most Windows/macOS apps, etc.
Containers, and virtualization based technologies.

Wait, won’t build + deployment be complex due to monolithic?

::: { .fragment } Chunking. :::

Break big software into multiple parts.
- Inside each part, we go monolithic.
- Between parts, apply simpler dependency management. :::

Break complex build and deployment into multiple stages.
- Inside each stage, we go imperative.
- Between xstages, we declaratively define dependencies. :::

Break huge docker image into multiple layers.
- Inside each layer, we go imperative.
- Between layers, apply simpler dependency management. ::: :::

What is chunking?

Imagine you need to data structure with both fast random access, and fast random insertion/deletion, and you find Balanced Trees to hard to implement.
Notice that:
- Arrays have O(1) random access, but O(N) random deletion / insertion.
- Linked lists have O(N) random access, but O(1) random deletion / insertion.
Partition N data into $\sqrt{N}$ $N$ chunks.
- Use linked list between chunks, and array within each chunk.
- $O(\sqrt{N})$ for both operations. :::

But we sacrifice sharing between components.

How many Electrons do you have in Windows/macOS?
How many Linux base images do you have in your K8s cluster?

There are optimizations, but coarse-grained.
Split into layers and share common bases. :::

Idea #3

Can we marry isolation with fine-grained package management?

Environment issues:
- Do we really need SAT model for dependency management?
- Much simpler if dependency is just a deterministic tree.
Managability issues:
- Store components isolately without bundling?
- Let components see dependencies in separate views? :::

Nix

Originating from The Purely Functional Software Deployment Model (2006),

Nix solves all above problems,

while achieving decalrative builds & deployments.

# Works for any Linux distribution and macOS
curl https://nixos.org/nix/install | sh

Nix: build system

Deterministic software building makes dependencies deterministic.

$\text{Inputs} \xrightarrow{f} \text{Output}$

::: { .r-stack } ::: { .fragment .fade-out data-fragment-index=“1” }

$x = 1, y = 2 \xrightarrow{f(x, y) = x + y} 1 + 2 = 3$

::: ::: { .fragment .current-visible data-fragment-index=“1” }

$\text{gcc, libc, source, ...} \xrightarrow{\text{./configure, make, make install}} \text{binary}$

Derivation

In Nix, we call the smallest unit of compliation as derivation.
A derivation is written in a Purely Functional Programming Language (Nix).
A derivation must have all inputs explictly specified.
A derivation is built inside a sandbox to avoid global state.
Therefore, the build output of a derivation is deterministic. :::

Nix store

Derivations are stored in Nix Store, and builds output to Nix Store.
A derivation’s build inputs are also managed in Nix Store.

Nix store paths made unique with cryptographic hash:

/nix/store/zrwzkd3szh13zd3wrlzj0kdkgiv1xzjn-hello.drv
/nix/store/rq6w0k38h7kbh2s9snwpysk5yph2fqbf-hello

The output path’s hash is generated by derivation content.
- Any slight change to build process is reflected in hash.
Nix store is read-only, only modifiable by nix. :::

Sandboxing

Builds only see specified inputs, and no other files.
- Assume nothing in global paths like /lib, /usr/bin, etc.
Private version of /proc, /dev, /dev/shm and /dev/pts (Linux-only).
- Therefore, private PID, mount, IPS, UTS namespace, etc.
- No networking access during build. :::

::: { .r-stack } ::: { .fragment .fade-out data-fragment-index=“1” }

An example “hello world” program:

/* ./src/main.c */
#include <stdio.h>

int main() {
    printf("Hello, World!");
    return 0;
}

… and how we write derivation for it:

{ pkgs, ... }:

pkgs.stdenv.mkDerivation {
  name = "hello";
  src = ./src;

  nativeBuildInputs = with pkgs; [ gcc ];
  buildPhase = ''
    gcc main.c -o hello
  '';
  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin
  '';
}

{
  "/nix/store/87zf1q5dx3dkn597lqq17f1g83y116l6-hello.drv": {
    "args": [
      "-e",
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh"
    ],
    "builder": "/nix/store/0c337gsdfjf3162avbkchh0yh4qbs2s3-bash-5.2-p15/bin/bash",
    "env": {
      "buildPhase": "gcc main.c -o hello\n",
      "builder": "/nix/store/0c337gsdfjf3162avbkchh0yh4qbs2s3-bash-5.2-p15/bin/bash",
      "installPhase": "mkdir -p $out/bin\ncp hello $out/bin\n",
      "name": "hello",
      "nativeBuildInputs": "/nix/store/hc326d04c91h73ndqdx3qkggsk730kf2-gcc-wrapper-12.3.0",
      "out": "/nix/store/8ygc7ks9ggj7p2q0b98w1axc3mkyi68c-hello",
      "outputs": "out",
      "src": "/nix/store/92m3yxqi2hfmj75b053zvj0kkhv9bplq-src",
      "stdenv": "/nix/store/iszb73m627pq8v3gwf7zl6xaw01ln2hj-stdenv-linux",
      "system": "aarch64-linux"
    }
    // to be continued...
  }
}

::: ::: { .fragment }

{{ // continuing
    "inputDrvs": {
      "/nix/store/f27pfz65b77lby39rrr48ps21pa6mbxj-gcc-wrapper-12.3.0.drv": {
        "outputs": [ "out" ]
      },
      "/nix/store/hq9032m10smw5qbig1b1cvvqirv61j54-stdenv-linux.drv": {
        "outputs": [ "out" ]
      },
      "/nix/store/nsn38mpj8j5h9861w5chg40f2vz4blq3-bash-5.2-p15.drv": {
        "outputs": [ "out" ]
      }
    },
    "inputSrcs": [
      "/nix/store/92m3yxqi2hfmj75b053zvj0kkhv9bplq-src",
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh"
    ],
    "name": "hello",
    "outputs": {
      "out": { "path": "/nix/store/8ygc7ks9ggj7p2q0b98w1axc3mkyi68c-hello" }
    },
    "system": "aarch64-linux"
  }
}

::: { .r-stack } ::: { .fragment .fade-out data-fragment-index=“1” }

Another example of hello world with ncurses

/* ./src/main.c */
#include <ncurses.h>

int main() {
    initscr();
    printw("Hello, World!");
    refresh();
    getch();
    endwin();
    return 0;
}

… ncurses needs to be explictly specified as buildInputs:

{ pkgs, ... }:

pkgs.stdenv.mkDerivation {
  name = "hello";
  src = ./src;

  nativeBuildInputs = with pkgs; [ gcc ];
  buildInputs = with pkgs; [ ncurses ];
  buildPhase = ''
    gcc main.c -o hello -lncurses
  '';
  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin
  '';
}

{
  "/nix/store/2w3jmr0s30ylyvpri0m2kb91q4c6wvcb-hello.drv": {
    "args": [
      "-e",
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh"
    ],
    "builder": "/nix/store/0c337gsdfjf3162avbkchh0yh4qbs2s3-bash-5.2-p15/bin/bash",
    "env": {
      "buildInputs": "/nix/store/k7wlgnj0d7fp3862gy0s5s6vphkm48k1-ncurses-6.4-dev",
      "buildPhase": "gcc main.c -o hello -lncurses\n",
      "builder": "/nix/store/0c337gsdfjf3162avbkchh0yh4qbs2s3-bash-5.2-p15/bin/bash",
      "installPhase": "mkdir -p $out/bin\ncp hello $out/bin\n",
      "name": "hello",
      "nativeBuildInputs": "/nix/store/hc326d04c91h73ndqdx3qkggsk730kf2-gcc-wrapper-12.3.0",
      "out": "/nix/store/rmkrazrqfy8zpk1h52qnjrj9qlcyh9mv-hello",
      "src": "/nix/store/yf2fijnfz19kqh8finky3n2rk11217r9-src",
      "stdenv": "/nix/store/iszb73m627pq8v3gwf7zl6xaw01ln2hj-stdenv-linux",
      "system": "aarch64-linux"
    }
  }
}

::: ::: { .fragment }

{{ // continuing
    "inputDrvs": {
      "/nix/store/5l9mg0nlx3j0nf08hlaspnnx592acfm1-ncurses-6.4.drv": {
        "outputs": [ "dev" ]
      },
      "/nix/store/f27pfz65b77lby39rrr48ps21pa6mbxj-gcc-wrapper-12.3.0.drv": {
        "outputs": [ "out" ]
      },
      "/nix/store/hq9032m10smw5qbig1b1cvvqirv61j54-stdenv-linux.drv": {
        "outputs": [ "out" ]
      },
      "/nix/store/nsn38mpj8j5h9861w5chg40f2vz4blq3-bash-5.2-p15.drv": {
        "outputs": [ "out" ]
      }
    },
    "inputSrcs": [
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh",
      "/nix/store/yf2fijnfz19kqh8finky3n2rk11217r9-src"
    ],
    "name": "hello",
    "outputs": {
      "out": { "path": "/nix/store/rmkrazrqfy8zpk1h52qnjrj9qlcyh9mv-hello" }
    },
    "system": "aarch64-linux"
  }
}

::: { .r-stack } ::: { .fragment .fade-out data-fragment-index=“1” }

What if we call other binaries in source?

#include <stdlib.h>

int main() {
    system("cowsay 'Hello World!'");
    return 0;
}

Create a wrapper to set PATH before actually executing the binary

{ pkgs, ... }:

pkgs.stdenv.mkDerivation {
  name = "hello";
  src = ./src;

  nativeBuildInputs = with pkgs; [ gcc makeWrapper ];
  buildPhase = ''
    gcc main.c -o hello
  '';
  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin
  '';
  postFixup = ''
    wrapProgram $out/bin/hello \
      --prefix PATH : "${pkgs.lib.makeBinPath [ pkgs.cowsay ]}"
  '';
}

/bin/hello is a wrapper script instead of real binary

$ cat /nix/store/z7i77wwagy58f6svxc8ksm5snsc8wnrm-hello/bin/hello

#! /nix/store/0c337gsdfjf3162avbkchh0yh4qbs2s3-bash-5.2-p15/bin/bash -e
PATH=${PATH:+':'$PATH':'}
PATH=${PATH/':''/nix/store/klqwsfd2xn14bb977d5dvjqdjpp6ka74-cowsay-3.7.0/bin'':'/':'}
PATH='/nix/store/klqwsfd2xn14bb977d5dvjqdjpp6ka74-cowsay-3.7.0/bin'$PATH
PATH=${PATH#':'}
PATH=${PATH%':'}
export PATH
exec -a "$0" "/nix/store/z7i77wwagy58f6svxc8ksm5snsc8wnrm-hello/bin/.hello-wrapped"  "$@"

Binary cache

Note that the output hash can be calculated without building the derivation.
Meaning, we can cache the builds easily.
Serve nix store with a file server, which is the binary cache.
Packages can be signed before being added to binary cache or on the fly as they are served. :::

NixPkgs

There are many languages/frameworks, complicated.

Why not unite the efforts in building software?
Nixpkgs provides not only compiler toolchains, but also infrastructure for packaging applications written in various language and frameworks.
And of course, it comes with an official binary cache.

It seems to restrictive for majority packages to onboard?

Nix Search - Packages

Repology

Take rust-analyzer as a real-life example.

Nix: package manager

Build makes little sense if we cannot install it.

Search for references recursively for a package’s derivation,
we get a package’s closure,
that is, itself and all its direct and transitive runtime dependencies.

Installation

Two possibilities:

The closure is already in store or binary cache,
It has to be built fresh (then we infer the closure).

::: { .fragment data-fragment-index=“1” } How to make the package accessible?

Activation script: an idempotent script making things accessible.
e.g. add all package outputs to $PATH.
Executed while initializing profile.
```
nix-shell -p hello
```
:::

Removal

Garbage collection.

Register it as gcroot when installing a derivation.
Deregister after uninstalling.
Periodically, enumerate all reachable store paths from the gcroots, and remove all unreachable paths.
```
$ nix-collect-garbage
```
:::

Advantages

Deterministic dependencies. No SAT solver.
Different versions / variants of the same package can be installed together.
Zero assumption about system global state.
Atomic installs / upgrades. :::

NixOS

::: { .fragment .semi-fade-out data-fragment-index=“1” } A working system = Software + Configurations :::

::: { .fragment data-fragment-index=“1” } Why separate packages and configurations?

NixOS leverages Nix to manage both altogether. :::

Filesystem Hierarchy Standard

/
├── boot
├── bin
├── etc
├── lib
├── usr
│  ├── bin
│  ├── include
│  ├── lib
│  └── share

However, in NixOS …

/
├── boot
├── nix
│  ├── store
│  │  ├── acr...kl-nixos-system-...
│  │  ├── 11w...cv-nixos-system-...
│  │  │  ├── activate
│  │  │  ├── kernel -> /nix/store/bcd...3h-linux-6.6.30/bzImage
│  │  │  ├── systemd -> /nix/store/9cx...8s-systemd-255.4
│  │  │  ├── ...
│  │  ├── bcd...3h-linux-6.6.30
│  │  ├── 9cx...8s-systemd-255.4
│  │  ├── ...
│  ├── var/nix/profiles
│  │  ├── system -> system-250-link
│  │  ├── system-249-link -> /nix/store/acr...kl-nixos-system-...
│  │  ├── system-250-link -> /nix/store/11w...cv-nixos-system-...

Activation script: /nix/var/nix/profiles/system/activate

Declaratively describe your system in Nix:

{ config, lib, pkgs, ...}: {
	# Use systemd-boot EFI bootloader
	boot.loader.systemd-boot.enable = true;
	# Kernel configurations
	boot.supportedFilesystems = [ "bcachefs" ];
	boot.initrd.availableKernelModules = [ "xhci_pci" "sr_mod" ];
  	boot.kernelPackages = pkgs.linuxPackages_latest;
	# Use zsh for shell
 	programs.zsh.enable = true;
	programs.zsh.enableCompletion = true;
	# Enable the OpenSSH daemon.
  	services.openssh.enable = true;
	# Configure users
	users.users.codgi = {
		name = "codgician";
		home = "/home/codgi";
		shell = pkgs.zsh;
		openssh.authorizedKeys.keys = [ "..." ];
	};
}

Nixpkgs also provide configuration modules, search here.
How I manage my devices: github:codgician/serenitea-pot.
Demo.

The power of NixOS roots in the Nix language.

Validate configurations before deployment.
Reference values accross modules of configurations.
Effeciently reuse configurations.
Unified language interface for any system component.

And even more

Trivial remote deployment: just push closure over ssh
Home manager: Manage dotfiles under /home declaratively using Nix.
Impermanence: Trivially handle persistent states on systems with ephemeral root storage

Ending

Although I am a Nix enthusiastic, I still realize:
- Nix is hard to adopt in industry due to steep learning curve.
- Even if Nix has a novel model, implementations consist many workarounds: sometimes you “hack” to make a derivation work on Nix.
- Many existing infra may be “acceptable” enough. - Completely refactor everything with Nix is usually not necessary. :::

::: { .fragment .semi-fade-out data-fragment-index=“1” } Nix/NixOS is still gaining increasing visibility and popularity.

Google trends: NixOS

::: { .fragment data-fragment-index=“1” } Embracing Nix without fully switching

Thank you!

To get started, or learn further about Nix/NixOS:

Official website: nixos.org
NixOS manual: nixos.org/manual/nixos/stable
Nix docs: nix.dev
NixOS wiki: wiki.nixos.org
Search packages / configurations: search.nixos.org
Search for functions in nix (lang): noogle.dev

Slides are

generated by pandoc,

rendered by reveal.js,

and managed by Nix.

Fully open-sourced.

References

Supplementaries

Slides not shown by default

Wait, for hello-ncurses, doesn’t gcc do dynmaic linking by default?

Nix has a patched version of dynamic linker in stdenv.
- It never searches global library directories, like /lib, /usr/lib, etc.
- The linker adds -rpath flag for every library directory mentioned through -L flags.
Won’t this possibly include unnecessary dependencies?
- We don’t know in advance if a library is actually used by the linker.
- Leverage fix up stage at the end to patchelf and shrink rpath. :::

Wait, how can Nix magically know runtime dependencies?

“Runtime dependencies must be a subset of build time dependencies”.

Build time dependencies are explictly specified.
Runtime dependencies are automatically inferred by:
- Serializing store paths into NAR,
- Then search for references to other store paths within it. :::

How can this be true and how can it work?

It just works! 🤪

Wait, what if I have secrets in my configurations?

Nix store is globally readable, any user has R/O access.

Store encrypted secrets in Nix Store.
Decrypt secrets with user private key / host private key before service load.
Example solutions: agenix | sops-nix. :::