Safely build docker images with git svn ssh private keys

Often when building a docker container, your Dockerfile needs to pull down some code from a privately secured source control system like Git or SVN via SSH. In order to do this, the build process will need a valid ssh configuration local to the build that includes the necessary components to make this work (such as a known_hosts file, private key and ssh config file etc)

For example, a typical Dockerfile that needs to fetch code from SVN over SSH might contain lines like this:

RUN mkdir -p /root/.ssh
ADD ssh/my-private.key /root/.ssh/id_rsa
ADD ssh/my-known_hosts /root/.ssh/known_hosts
RUN yum -y install && \
    openssh-clients && \
    subversion && \
    chown -R root.root /root/.ssh/* && \
    chmod 600 /root/.ssh/* && \
    cd /targetdirforcode && \
    svn checkout svn+ssh:///mysvnhost/something/code && \
    rm -rf ~/.ssh && \
    yum -y remove subversion openssh-clients && \
    yum -y autoremove && \
    yum clean all

Whats the problem with this? Well each line in a Dockerfile is converted into a separate layer, and if you were to push the resulting image into a image registry, you would end up exposing the private key, despite the stanza that technically removes it (or seems to!) from the next layer of the cake. Each layer is treated separately in Docker image builds.

To get around this you could docker-squash your image before publishing it, however if part of a larger process or workflow on a development team, this squash add-on step has the chance of being forgotten and the private key inadvertently published in the resulting image push. There is nothing inherently “built in” to the Dockerfile except above to prevent this.

Possible solutions:

If you search around on this topic you will find a few different solutions

We looked at a few of these and were not really satisfied. We really wanted some process that the Dockerfile would not build and produce anything unless the key was delivered safely AND devops/developers only needed to have Docker installed to do it.

How we ended up doing it:

1. Configure your build machine

mkdir -p ~/mykeyserverroot/svn
cp my-svn.id_rsa ~/mykeyserverroot/svn/id_rsa
cp known_hosts ~/mykeyserverroot/svn/known_hosts
chmod 600 ~/mykeyserverroot/svn/*

2. Create build bridge network and start server

Note nginx below is not published on any local ports, it only exists on the bridge network and is accessible by the name buildkeyserver

docker network create my-build-net
 
docker run --name buildkeyserver \
 --network my-build-net \
 --network-alias buildkeyserver \
 -v ~/mykeyserverroot:/usr/share/nginx/html:ro \
 -d nginx

3. Adjust your Dockerfile to pull from ‘buildkeyserver’

RUN yum -y install && \
    openssh-clients && \
    subversion && \
    curl && \
    mkdir -p /root/.ssh && \
    curl -v -s -o /root/.ssh/id_rsa http://buildkeyserver/svn/id_rsa && \
    curl -v -s -o /root/.ssh/known_hosts http://buildkeyserver/svn/known_hosts && \
    chown -R root.root /root/.ssh/* && \
    chmod 600 /root/.ssh/* && \
    cd /targetdirforcode && \
    svn checkout svn+ssh:///mysvnhost/something/code && \
    rm -rf ~/.ssh && \
    yum -y remove curl subversion openssh-clients && \
    yum -y autoremove && \
    yum clean all

4. Build your image

docker build --network my-build-net -t my-app:1.0  .
 
docker rm -f buildkeyserver

The advantage of the above is that its simple, requires no special software to exist and be configured on the build machine and retains the image layering history. The example above can just as easily adapted for Git etc. You can really leverage this model for any need where sensitive data is needed at build time.

View all posts by bitsofinfo

Originally published at bitsofinfo.wordpress.com on November 30, 2017.