Build secrets and SSH forwarding in Docker 18.09

One of the complexities when using Dockerfiles has always been accessing private resources. If you need to access some private repository or service there really wasn’t a very good solution to achieve that. You shouldn’t use environment variables or plainly remove the secret files after use because they would still remain in the metadata of the image. Some creative use cases leveraged multi-stage builds, but the user still needed to be very careful to make sure the final stage is clean from all secure values, and the secret files would be kept in the local build cache until it is pruned.

The build command Docker 18.09 comes with a lot of new updates. Most importantly it can now use a completely new backend implementation that is provided by the Moby BuildKit project. BuildKit backend comes with a bunch of new features, one of them being build secrets support in Dockerfiles.

Using secrets

The first thing to do to use build secrets is to enable BuildKit backend. BuildKit is an opt-in feature in 18.09 that can be enabled with an environment variable DOCKER_BUILDKIT=1before running docker build. The intention is to make BuildKit backend default in a future release.

export DOCKER_BUILDKIT=1

Build secrets implementation is based on two new features provided by BuildKit. One is the ability to use custom builder frontends loaded from images in the registry, and the other is the possibility to use mounts in RUN commands for Dockerfiles. To use the implementation with secrets support enabled instead of the default one, you need to define the builder image with a syntax directive as a first line of the Dockerfile, pointing to the container image you wish to use. Secrets are currently not enabled in the stable channel of external Dockerfiles, so you need to use one of the releases in the experimental channel, e.g. docker/dockerfile:experimental or docker/dockerfile/1.0.0-experimental.

# syntax=docker/dockerfile:1.0.0-experimental

As a Dockerfile author, when you know that a RUN command defined in your Dockerfiles needs to use a secret value, you should use a --mount flag on that command, specifying what secret the command needs and where to mount it. The mount flag accepts a comma-separated structure similar to the --mount flag when using docker run.

# syntax=docker/dockerfile:1.0.0-experimental
FROM alpine
RUN --mount=type=secret,id=mysite.key command-to-run

That flag defines that when this command runs, it has access to a secret file on the path /run/secrets/mysite.key. The secret is only exposed to the single command that defined the mount, not to the other parts of the build. The data that this file contains is loaded from a secret store based on the specified ID “mysite.key”. Docker CLI currently supports exposing secrets from local files from the client using the --secret flag.

docker build --secret id=mysite.key,src=path/to/mysite.key .

As described earlier the secrets are by default mounted to /run/secrets but you can specify any path you want using the “target” key. If you specify “target” but no “id”, then the “id” defaults to the basename of the target path.

You are not limited to using a single secret and can use as many as you want by specifying different IDs.

If the Dockerfile author defines that a RUN instruction can use a secret but the user invoking a build does not provide it, the secret is ignored and no file is mounted to the path. If this is not the behavior that you want, you can use the “required” key to mark that a build should fail if no value is passed.

# syntax=docker/dockerfile:1.0.0-experimental
FROM alpine
RUN --mount=type=secret,id=mysite.key,required <command-to-run>

Implementation

A secret file is automatically mounted only to a separate tmpfs filesystem to make sure that it does not leak to the final image nor to the next command and that it does not remain in the local build cache.

The secret values are also excluded from the build cache calculations to avoid anyone from using the cache metadata from getting information about the secret value.

SSH

Probably the most popular use case for accessing private data in builds is for getting access to private repositories through SSH. Even though you could use secret mounts to expose the private SSH key to the build, we thought we could do better. SSH uses public key cryptography and based on that design you are not really supposed to share your private keys with anyone. For example, when using multiple machines with SSH, you don’t transfer your keys but forward your connection with ssh -A.

We have added a similar possibility into docker build where you can use the--ssh flag to forward your existing SSH agent connection or a key to the builder. Instead of transferring the key data, docker will just notify the builder that such capability is available. Now when the builder needs access to a remote server through SSH, it will dial back to the client and ask it to sign the specific request needed for this connection. The key itself never leaves the client, and as soon as the command that requested the access has completed there is no information on the builder side to reestablish that remote connection later.

Only the commands in the Dockerfile that have explicitly requested the SSH access by defining a type=ssh mount get access to SSH forwarding. The other commands have no knowledge of any SSH agent being available.

Another aspect to note about SSH is that it uses a trust-on-first-use security model. When you are connecting to an SSH server for the first time, it will prompt you about an unknown host, because it doesn’t have the public key of that server locally available, and therefore can’t check if the public key provided by the remote party is valid for that address.

If you are performing builds with a Dockerfile you cannot verify the correctness of that prompt, and therefore the public key of the server must already exist on the container that is trying to use SSH. There are a couple of ways how you could get that public key. For example, the base image could provide it or you might want to copy it from your build context. If you want a more straightforward solution, you can run thessh-keyscan program as part of your build that downloads the current public key for that host for you.

To request SSH access for a RUN command in Dockerfile you need to define a mount with type “ssh”. That means that a socket with a read-only access to the SSH agent will be mounted while that process runs. This will also set up the SSH_AUTH_SOCK environment variable to make programs relying on SSH automatically use that socket.

# syntax=docker/dockerfile:experimental
FROM alpine
# install ssh client and git
RUN apk add --no-cache openssh-client git
# download public key for github.com
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
# clone our private repository
RUN --mount=type=ssh git clone git@github.com:myorg/myproject.git myproject

On the docker client side, you need to define that SSH forwarding is allowed for this build by using the--ssh flag.

docker build --ssh default .

The flag accepts a key-value pair defining the location for the local SSH agent socket or the private keys. The socket path can be left empty if you want to use the value of default=$SSH_AUTH_SOCK. Note that when using the default configuration you need to add your keys to your local SSH agent and we will not connect your ~/.ssh/id_rsa key automatically. You can checkssh-add -L locally to see if the public keys are visible to the agent.

The mount in the Dockerfile can also use an “id” key to separate different servers that are part of the same build. For example, the different repositories in your Dockerfile may be accessible with different deploy keys. In this case, in Dockerfile you would use:

…
RUN --mount=type=ssh,id=projecta git clone projecta …
RUN --mount=type=ssh,id=projectb git clone projectb …

… and expose it from the client with docker build --ssh projecta=./projecta.pem --ssh projectb=./projectb.pem. Note that even when actual keys are specified here, still only the agent connection is shared with the builder and not the actual content of these private keys.

This concludes the overview of the new build secrets capabilities in Docker 18.09. I hope these new features help you take advantage of the power of Dockerfiles better in your private projects and make your build pipeline more secure.