Rust: How to build a Docker image with private Cargo dependencies
Rust is growing in popularity, having earned the most loved language in Stack Overflow’s 2019 Developer Survey for the fourth year in a row. While Cargo and Rust’s tooling are great, there’s still a few tricks in getting them to work in a production setting
In this article, I’m going to show you how to fetch private Cargo dependencies and source them when building a Docker image. This solves a key issue with Docker of not copying over SSH keys when building an image
The code for this blog post can be found in the Github repository below
Example Application: Rocket-Add
I’ve developed an example API called Rocket-Add which is built on top of a very cool web framework in Rust called Rocket
This application extends the ‘Hello World’ example with a new API call add/<num1>/<num2>
which adds two numbers and outputs the result
$ curl localhost:8000/add/5/10
The sum of 5 and 10 is 15
The API endpoint is written as a function
Looking closely, you can see the addition calls an external library math_utils
with the call math_utils::add(num1, num2)
The math_utils
crate is from a private Github repository https://github.com/c-ameron/rust-math-utils.git
For reference, this is the code for the add
function in math_utils
being used
Fetching crates in private repositories
Before we go any further, we’re going to take a page from Ruby Bundler’s book and create a local .cargo
folder in the project root. This will provide a project-independent place to store our crates and Cargo config
git clone git@github.com:c-ameron/rocket-add.git
cd rocket-add
mkdir -p $(git rev-parse --show-toplevel)/.cargo
export CARGO_HOME=$(git rev-parse --show-toplevel)/.cargo
Currently Cargo by default won’t fetch from private repositories
To get around this, create a .cargo/config
file which tells Cargo to use the git cli for fetching
$ cat .cargo/config
[net]
git-fetch-with-cli = true
Then in our Cargo.toml
we tell Cargo to fetch our crate with SSH
math_utils = { version = "0.1.0", git = "ssh://git@github.com/c-ameron/rust-math-utils.git"}
Now we can run cargo fetch
and it will download all the crates to the local .cargo
folder
Building a Rust Docker image with private dependencies
A frequently encountered problem when building a Docker image is downloading private dependencies without mounting an SSH key
To solve this problem, we can copy over our pre-fetched .cargo
folder to the Docker image and build from there. Now Cargo won’t need to use any SSH keys to fetch from private repositories
Note that we’re having to set the WORKDIR
and CARGO_HOME
inside the Docker image for Cargo to resolve its crates correctly
We can also take advantage of multi-stage builds in Docker, and create a smaller binary image
This approach saves time downloading the same crates each build and we also don’t need to pass any SSH keys to Docker. However, if there’s any change to our dependencies it will require a rebuild. For a larger application with a lot of changing dependencies, this often means rebuilding from scratch several times a day
Notes
- Docker has implemented experimental support for SSH forwarding when building images in 18.09, potentially negating the need for copying
.cargo
in the future - If you want to fetch or build an image completely offline with Cargo, you can use the
-Z offline
flag. See https://doc.rust-lang.org/cargo/reference/unstable.html#offline-mode - With recently released Rust 1.34, Cargo now has support for alternative registries other than
crates.io