The Nix Fix
Once upon a time, there was a docker container.
Written with epic support from our nix master, Ding Xiang Fei (xiangfei@tenx.tech)
It sounded like the best idea in the world. I remember setting up Docker for my team at a previous venture because we were starting to run into all kinds of version discrepancies during development.
So I dockerized our dev stack. “Should take me a day or two, max”, I confidently reassured my team.
How long did it actually take me?
Hint: The best algorithm I’ve learned so far for transforming a technical time estimate, including my own, into a measure of reality, is to double the estimate’s number, and bump its time unit by an order of magnitude.
Here’s what the result approximately looked like. A typical Dockerfile for local development. How many potential impurities can you detect? Do you think it would still build the same if you ran it today?
FROM node:6.3.1# gosu
ENV GOSU_VERSION 1.9
RUN set -x \
&& apt-get update && apt-get install -y --no-install-recommends ca-certificates wget \
&& dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc" \
&& export GNUPGHOME="$(mktemp -d)" \
&& gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
&& gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
&& rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& gosu nobody true# CHIMP
# Dependency: Google Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
RUN apt-get update
RUN apt-get install -y google-chrome-stable libexif-dev# Dependency: Git
RUN apt-get install -y git-core# Dependency: xvfb (fake screen)
RUN apt-get install -y xvfb# X11VNC
RUN apt-get install -y x11vnc
RUN mkdir ~/.vnc
RUN x11vnc -storepasswd secretchimpatee ~/.vnc/passwd# Adding local user
RUN adduser --disabled-password --gecos '' meteor
# Setting up directories and perms
RUN mkdir /home/meteor/ci \
&& chown -R meteor:meteor /home/meteor# Initializing named volumes
RUN mkdir /home/meteor/.meteor \
&& chown -R meteor:meteor /home/meteor/.meteorRUN mkdir -p /home/meteor/web/.meteor/local \
&& chown -R meteor:meteor /home/meteor/webUSER meteor# Installing METEOR
RUN curl https://install.meteor.com/ | sh
# Symlinking METEOR executable as root
USER root
RUN ln -s /home/meteor/.meteor/meteor /usr/local/bin/meteor
USER meteorWORKDIR /home/meteor/ci# Install chimp's NPM dependencies
COPY package.json .
RUN npm install# Cache Chimp's Auto-Installed Dependencies
RUN node_modules/.bin/chimp --path=git-hooksCOPY .scripts/start.js ./.scripts/
COPY .scripts/headless-start.js ./.scripts/
COPY data ./data/
COPY chimp.js .
COPY chimp-ci.js .CMD ["node", ".scripts/headless-start.js"]
Docker has definitely matured since my first no-longer-bare-knuckled fistfights with it four years back in time. But some inherent design flaws can’t be easily fixed, and they were slowing TenX’s epic quest towards decentralized teams distributed systems.
The Dangers of Docker
- Images built may be non-reproducible
- Performance, especially on our mac file systems, can be prohibitive
- As a result of nondeterministic builds, orchestration tools are unnecessarily complicated
- The image you build tomorrow may very well not be the image you created with the same set of instructions today. Yes, this is what happened to point 1 after time and impure inputs (lack of coffee) affected my sentence build process. Nevermind that.
Once upon a time, there was a docker container. We tried to rebuild it a year later, and it looked different. Verflixt!
Building PostgreSQL in Docker
Here’s a classic example:
FROM alpine:3.6RUN apk update && apk add postgresql
We hope it gets us to the right version of PostgreSQL, built with the right libraries in their correct versions, but we are under the mercy of the package maintainers from alpine.
It’s simple enough for most cases because we just want the latest versions of everything, but this kind of efficiency shortcut only holds up until it doesn’t.
Good luck hunting down the rabbit hole of dependency hell for troubleshooting a problem after bumping the version just slightly or, even worse, when you want to re-trigger this build a few months later.
Source of trust for Docker: The Docker registry, the image maintainer and their promise to build the images strictly according to their published Dockerfiles.
Docker does not guarantee true determinism since many images are built against resources obtained from the network.
…vs Building PostgreSQL in Nix
with import ./my-nixpkgs {};
dockerTools.buildImage {
name = “postgresql-docker-image”;
tag = “latest”;
contents = [ postgresql ];
}When evaluated, this will produce a nix derivation looking something like this:
«derivation /nix/store/p76wh1pri1c2xmpzn1j066b04318z7a5-docker-image-postgresql-docker-image.tar.gz.drv»Note the cryptographic hash `p76wh1pri1c2xmpzn1j066b04318z7a5` in the output.
There’s a couple of things you may note here.
- You can choose your own revision of
nixpkgs, likemy-nixpkgsfor instance - The build process is predictable, as every dependency leading up to the result is verified by cryptographic hashes and, even better,
all build processes of these dependencies are themselves declaratively defined, transparent and, thus, reproducible. - Developers can go the extra mile to ensure determinism by building their software in Nix’s sandbox mode, which shields all system impurities,
network and external devices away from the build process, so as to achieve truly reproducible builds.
Source of trust for Nix: your nixpkgs of choice, cryptographic hashes, Github’s commit history, and transparent build recipes.
Contributing to the Nix ecosystem
At TenX engineering, we believe that nix is making the right design decisions, even though it still has its rough edges. From simple security updates to fixing cross-compilation issues all the way to improving the toolchain, we have hackers of all proficiency levels contributing to the Nix ecosystem.
At the time of writing, we have 2 open PRs against nixpkgs:
Want to help? Please do!
Don’t know where to start? Code triage would be a great start, or writing excellent documentation!
Time Travel with Nix
In its essence, nix is a purely functional package manager, and as such it will force you to package your software with time travel, i.e. immutability in mind.
This can become hard at times, and so nix hands you a number of tools to ease the task.
One of these tools is nix itself, which happens to be a pure, lazy, functional language in which you can do much more than mere package description.
The syntax looked a little unfamiliar to me at first, but I’m starting to get the hang:
((x: (y: (x*x + y*y))) 3) 7Can you work this out? Here’s a great post to get started with nix, the language:
https://medium.com/@MrJamesFisher/nix-by-example-a0063a1a4c55
— —
At TenX, we love functional.
At TenX, we also love time travel. We need to. We are now deploying production-ready code multiple times a day — things can and will go wrong. We can’t have that affect our users in any critical way. So when we rollback a change, and that change requires a different version of a library, we need that library to build precisely in the same way that it built an hour, a day, or even a year ago.
So why not have pure build functions?
That’s exactly what nix does for us.
/nɪks/ Austrian [slang]: “nothing”
Once upon a time, there was a docker container.
Nix came, nix built, nix changed.
