nix2container

nix2container provides an efficient container development workflow with images built by Nix: it doesn't write tarballs to the Nix store and allows to skip already pushed layers (without having to rebuild them).

This is based on ideas developed in this blog post.

Getting started

{
  inputs.nix2container.url = "github:nlewo/nix2container";
outputs = { self, nixpkgs, nix2container }: let
    pkgs = import nixpkgs { system = "x86_64-linux"; };
    nix2containerPkgs = nix2container.packages.x86_64-linux;
  in {
    packages.x86_64-linux.hello = nix2containerPkgs.nix2container.buildImage {
      name = "hello";
      config = {
        entrypoint = ["${pkgs.hello}/bin/hello"];
      };
    };
  };
}

This image can then be loaded into Docker with

$ nix run .#hello.copyToDockerDaemon
$ docker run hello:latest
Hello, world!

More Examples

To load and run the bash example image into Podman:

$ nix run github:nlewo/nix2container#examples.bash.copyToPodman
$ podman run -it bash

• bash: Bash in /bin/
• fromImage: Alpine as base image
• fromImageManifest: Alpine as base image, from a stored manifest.json.
• nginx
• nonReproducible: with a non reproducible store path :/
• openbar: set permissions on files (without root nor VM)
• uwsgi: isolate dependencies in layers
• layered: build a layered image as described in this blog post

Functions documentation

nix2container.buildImage

Function arguments are:

• name (required): the name of the image.

• tag (defaults to the image output hash): the tag of the image.

• config (defaults to {}): an attribute set describing an image configuration as

• copyToRoot (defaults to null): a derivation (or list of

pkgs.buildEnv can be used to build a derivation which has to be copied to the image root. For instance, to get bash and coreutils in the image /bin:

copyToRoot = pkgs.buildEnv {
      name = "root";
      paths = [ pkgs.bashInteractive pkgs.coreutils ];
      pathsToLink = [ "/bin" ];
    };

• fromImage (defaults to null): an image that is used as base

• fromImageEnv (defaults to false): keep the Env entries of

• maxLayers (defaults to 1): the maximum number of layers to

• perms (defaults to []): a list of file permisssions which are

Each element of this permission list is a dict such as

{ path = "a store path";
      regex = ".*";
      mode = "0664";
    }

mode sets the mode, and orMode adds bits to it. For instance, orMode = "0200"; makes the files writable by their owner and keeps their execute bits. With both, mode comes first: { mode = "0444"; orMode = "0200"; } gives 0644. The entries are applied in list order.

• initializeNixDatabase (defaults to false): to initialize the

• layers (defaults to []): a list of layers built with the

nix2container.pullImage

Pull an image from a container registry by name and tag/digest, storing the entirety of the image (manifest and layer tarballs) in a single store path. The supplied sha256 is the narhash of that store path.

Function arguments are:

• imageName (required): the name of the image to pull.

• imageDigest (required): the digest of the image to pull.

• sha256 (required): the sha256 of the resulting fixed output derivation.

• os (defaults to linux)

• arch (defaults to x86_64)

• tlsVerify (defaults to true)

nix2container.pullImageFromManifest

Pull a base image from a container registry using a supplied manifest file, and the hashes contained within it. The advantages of this over the basic pullImage:

• Each layer archive is in its own store path, which means each will download just once

• There is no Nix-specific hash, so it's possible update the base image by simply

With this function the manifest.json acts as a lockfile meant to be stored in source control alongside the Nix container definitions. As a convenience, the manifest can be fetched/updated using the supplied passthru script, eg:

nix run .#examples.fromImageManifest.fromImage.getManifest > examples/alpine-manifest.json

Function arguments are:

• imageName (required): the name of the image to pull.

• imageManifest (required): the manifest file of the image to pull.

• imageTag (defaults to latest)

• os (defaults to linux)

• arch (defaults to x86_64)

• tlsVerify (defaults to true)

• registryUrl (defaults to registry.hub.docker.com)

Authentication

If the Nix daemon is used for building, here is how to set up registry authentication.

1. docker login URL to whatever it is
2. Copy ~/.docker/config.json to /etc/nix/skopeo/auth.json
3. Make the directory and all the files readable to the nixbld group:

sudo chmod -R g+rx /etc/nix/skopeo
   sudo chgrp -R nixbld /etc/nix/skopeo

1. Bind mount the file into the Nix build sandbox

extra-sandbox-paths = /etc/skopeo/auth.json=/etc/nix/skopeo/auth.json

Every time a new registry authentication has to be added, update /etc/nix/skopeo/auth.json file.

nix2container.buildLayer

For most use cases, this function is not required. However, it could be useful to explicitly isolate some parts of the image in dedicated layers, for caching (see the "Isolate dependencies in dedicated layers" section) or non reproducibility (see the reproducible argument) purposes.

Function arguments are:

• deps (defaults to []): a list of store paths to include in the

• copyToRoot (defaults to null): a derivation (or list of

pkgs.buildEnv can be used to build a derivation which has to be copied to the image root. For instance, to get bash and coreutils in the image /bin:

copyToRoot = pkgs.buildEnv {
      name = "root";
      paths = [ pkgs.bashInteractive pkgs.coreutils ];
      pathsToLink = [ "/bin" ];
    };

• reproducible (defaults to true): If false, the layer tarball

• maxLayers (defaults to 1): the maximum number of layers to

• perms (defaults to []): a list of file permisssions which are

• permsFile (defaults to null): a JSON file holding the list

Each element of this permission list is a dict such as

{ path = "a store path";
      regex = ".*";
      mode = "0664";
    }

mode sets the mode, and orMode adds bits to it. For instance, orMode = "0200"; makes the files writable by their owner and keeps their execute bits. With both, mode comes first: { mode = "0444"; orMode = "0200"; } gives 0644. The entries are applied in list order.

• layers (defaults to []): a list of layers built with the

• ignore (defaults to null): a store path to ignore when

• metadata (defaults to { created_by = "nix2container"; }): an attribute

Isolate dependencies in dedicated layers

It is possible to isolate application dependencies in a dedicated layer. This layer is built by its own derivation: if storepaths composing this layer don't change, the layer is not rebuilt. Moreover, Skopeo can avoid to push this layer if it has already been pushed.

Let's consider an application printing a conversation. This script depends on bash and the hello binary. Because most of the changes concern the script itself, it would be nice to isolate scripts dependencies in a dedicated layer: when we modify the script, we only need to rebuild and push the layer containing the script. The layer containing dependencies won't be rebuilt and pushed.

As shown below, the buildImage.layers attribute allows to explicitly specify a set of dependencies to isolate.

{ pkgs }:
let
  application = pkgs.writeScript "conversation" ''
    ${pkgs.hello}/bin/hello
    echo "Haaa aa... I'm dying!!!"
  '';
in
pkgs.nix2container.buildImage {
  name = "hello";
  config = {
    entrypoint = ["${pkgs.bash}/bin/bash" application];
  };
  layers = [
    (pkgs.nix2container.buildLayer { deps = [pkgs.bash pkgs.hello]; })
  ];
}

This image contains 2 layers: a layer with bash and hello closures and a second layer containing the script only.

In real life, the isolated layer can contains a Python environment or Node modules.

See Nix & Docker: Layer explicitly without duplicate packages! for learning how to avoid duplicate store paths in your explicitly layered images.

Quick and dirty benchmarks

The main goal of nix2container is to provide fast rebuild/push container cycles. In the following, we provide an order of magnitude of rebuild and repush time, for the uwsgi image.

warning: this is quick and dirty benchmarks which only provide an order of magnitude

We build the container and push the container. We then made a small change in the hello.py file to trigger a rebuild and a push.

Method | Rebuild/repush time | Executed command ---|---|--- nix2container.buildImage | ~1.8s | nix run .#example.uwsgi.copyToRegistry dockerTools.streamLayeredImage | ~7.5s | nix build .#example.uwsgi \| docker load dockerTools.buildImage | ~10s | nix build .#example.uwsgi; skopeo copy docker-archive://./result docker://localhost:5000/uwsgi:latest

Note we could not compare the same distribution mechanisms because

• Skopeo is not able to skip already loaded layers by the Docker daemon and
• Skopeo failed to push to the registry an image streamed to stdin.

Run the tests

nix run .#tests.all

This builds several example images with Nix, loads them with Skopeo, runs them with Podman, and test output logs.

Not that, unfortunately, these tests are not executed in the Nix sandbox because it is currently not possible to run a container in the Nix sandbox.

It is also possible to run a specific test:

nix run .#tests.basic

The nix2container Go library

This library is currently used by the Skopeo nix transport available in this branch.

For more information, refer to the Go documentation.

Commercial support

For commercial support (customizations, image optimizations and best practices guidance, bug fixes), please contact nlewo at [email protected].