Building docker images with private python packages
Installing private python packages into Docker container can be tricky because container does not have access to private repositories and you do not want to leave trace of private ssh key in docker images.
This article presents two methods that can be used.
First is using intermediate docker image where private SSH key is deployed to intermediate image that is deleted afterwards. This method can be used for other purposes where access to private server is needed only during build time.
Second method utilizes deploy or access keys that can set for various source code hosting services such as Github or Bitbucket. Idea is to create key based read only access to repositories per specific need. This access can be removed later if needed.
Check docker-images-with-private-python-packages-example repository for example files.
1. Build docker image with intermediate image
This method passes private ssh key to intermediate docker image that would be deleted after creating image.
Python packages are downloaded into intermediate container and then copied and installed in final image. It would not be possible to access private repository from inside container.
FROM python:3.6 as intermediate
COPY requirements.txt /
RUN pip download -r /requirements.txt
COPY --from=intermediate /pip-packages/ /pip-packages/
RUN pip install --no-index --find-links=/pip-packages/ /pip-packages/*
- create intermediate image
- download python packages to intermediate image
- create new image
- copy python packages from intermediate image
- install downloaded packages
To create image following command can be used:
docker build --force-rm -t test-multi-stage-builds --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" .
Description of command arguments:
--force-rm - forces deleting intermediate images, even if build fails
-t test-multi-stage-builds - image name
--build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" - set build-time variable so private repository can be accessed from intermediate image
2. Build docker image with deploy keys
This method depends on adding deploy/access key to every private repository. It would be possible to access repository from container until key is deleted.
To create deploy private/public keys, following command can be used:
ssh-keygen -t rsa -b 4096 -C "test-with-deploy-keys" -f "./deploy_key" -N ""
This command would create two files: private
deploy_key and public
deploy_key.pub should be added to repository Settings > Deploy keys on GitHub repository settings or Settings > Access keys on Bitbucket.
docker build -t test-with-deploy-keys --build-arg SSH_PRIVATE_KEY="$(cat ./deploy_key)" -f Dockerfile-deploykeys .
- deploy_key would be visible in
docker historyand in container.
- with this approach no intermediate image is needed.
Include arguments under
build key, for example:
And here is an example of passing private key to
docker-compose build --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)"
Github repository with all files mentioned in this article: docker-images-with-private-python-packages-example
Inspiration and initial code from :