Running Rust microservice with Podman and on 2 architectures

8 min readFeb 16, 2023

Introduction

In my previous article on Exploring layered configuration and concurrency with Rust I’ve shortly mentioned that one of my recent objectives was to be able to run some Microservice developed in Rust on 2 architectures, i.e. x86_64 and aarch64. And besides that I wanted to continue (that is what I’ve used for more than 1 year now) to use Podman instead of Docker as containers engine and to have similar experience as with docker-compose, i.e. ability to start/tear down multiple microservices within their own network in easy fashion.

Overall diagram of desired state

Below is diagram representing final state to achieve, which will be described in this article:

Source code

Could be downloaded/cloned from: https://github.com/andrejusc/rust-example-multi-architecture

Setup of my environment

For this article — main system’s setup is exactly same as for mentioned article above, i.e. VM with running Ubuntu 22.04.1 LTS and Rust v1.67.0. And that is so far using x86_64 architecture.

Project structure and source code

Before going further — here is presented project’s tree structure, where Rust cargo was initialized within service folder and env folder was made outside of src folder:

$ tree -I 'target'
.
├── LICENSE
├── README.md
└── service
    ├── build.rs
    ├── Cargo.lock
    ├── Cargo.toml
    ├── env
    │   ├── logging-dev.yml
    │   ├── logging.yml
    │   ├── service-dev.yml
    │   └── service.yml
    ├── image
    │   ├── Dockerfile
    │   ├── Dockerfile.aarch64
    │   ├── Dockerfile.curl
    │   ├── podman-compose.yml
    │   └── toolchain
    │       └── Dockerfile.toolchain-aarch64
    └── src
        ├── custom_tracing_layer.rs
        └── main.rs

Final list of dependencies in cargo.toml file

Providing it here for quick review ahead of whole article reading

[dependencies]
async-graphql = "5.0.6"
async-graphql-derive = "5.0.6"
async-graphql-poem = "5.0.6"
chrono = "0.4.23"
config = "0.13.3"
poem = "1.3.54"
serde_json = "1.0.93"
tokio = { version = "1.25.0", features = ["macros", "rt-multi-thread"] }
tracing = "0.1.37"
tracing-subscriber = "0.3.16"

[build-dependencies]
copy_to_output = "2.0.0"
glob = "0.3.1"

Podman setup

My default Ubuntu setup doesn’t have any containers engine installed and to have Podman — I’m using such steps:

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/xUbuntu_$(lsb_release -rs)/Release.key \
  | gpg --dearmor \
  | sudo tee /etc/apt/keyrings/devel_kubic_libcontainers_unstable.gpg > /dev/null
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/devel_kubic_libcontainers_unstable.gpg]\
    https://download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/xUbuntu_$(lsb_release -rs)/ /" \
  | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:unstable.list > /dev/null
sudo apt-get update -qq
sudo apt-get -qq -y install podman

To verify that setup went fine — you could invoke:

$ podman version
Client:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.18.1
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

And additional check in regards to which version and manager of cgroups it uses:

$ podman info | grep -P "cgroupManager|cgroupVersion"
  cgroupManager: systemd
  cgroupVersion: v2

Adding podman-compose

You could add podman-compose either in system-wide fashion:

curl -o /usr/local/bin/podman-compose https://raw.githubusercontent.com/containers/podman-compose/devel/podman_compose.py
chmod +x /usr/local/bin/podman-compose

or for your effective user:

curl -o ~/.local/bin/podman-compose https://raw.githubusercontent.com/containers/podman-compose/devel/podman_compose.py
chmod +x ~/.local/bin/podman-compose

I’ve selected 2nd approach for my effective user. But podman-compose also requires pip dependency dotenv and that I’ve installed in such way:

$ wget https://bootstrap.pypa.io/get-pip.py
$ python3 get-pip.py
$ python3 -m pip install python-dotenv

After that — quick verification check to see it’s version as well as version of underlying podman:

$ podman-compose version
podman-compose version: 1.0.4
['podman', '--version', '']
using podman version: 4.3.1
podman-compose version 1.0.4
podman --version 
podman version 4.3.1
exit code: 0

Microservice in Rust to try

As my exploration of Rust capabilities and available libraries continues — I’ve decided to prepare sample GraphQL microservice to be based on async-graphql library and to utilize as well described in my previous article layered configuration together with Tokio async framework again.

Initially to build static executable for x86_64 architecture — same command as in previous article is used:

RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-gnu

Prepared microservice will expose nothing fancy, but just 2 queries, which you could explore fast in GraphiQL UI (and such is also beauty of used async-graphql library) once described below podman-compose configuration will run and hitting localhost:8080 address (you’ll see later in podman-compose configuration I’ve mapped container’s port 4000 to host’s 8080) in browser:

File Dockerfile for this microservice is such:

FROM scratch
# Such copy here is only for Dev-like environments. For other Envs - should be mounted explicitly.
COPY ../env /env
COPY ../target/x86_64-unknown-linux-gnu/release/service /service
CMD [ "/service" ]

and to build image out of it — I’ve used such command:

$ podman build -t rust-example-multi-arch -f image/Dockerfile .

But to test it in non-interactive UI mode — usually you’d need utility like curl somewhere and so I’ve decided to have another light container based on curlimage/curl:latest, but make it to run in some infinitive loop. And so I don’t need to have it on my host explicitly installed and also I could invoke curl command via podman. File Dockerfile.curl is such then:

FROM curlimages/curl:latest
# Make container to run forever once started
CMD [ "tail", "-f", "/dev/null"  ]

and to build image out of it — I’ve used such command:

$ podman build -f image/Dockerfile.curl -t curl-loop .

Combining into podman-compose configuration

Now having 2 images — here is podman-compose configuration to combine them together:

# See versions list here: https://docs.docker.com/compose/compose-file/compose-file-v3/
version: "3.8"
services:

  rust-example:
    image: rust-example-multi-arch
    container_name: rust-example
    hostname: rust-example
    ports:
      - "8080:4000"
    links:
      - curl-loop:curl-loop
    volumes:
      - ../env:/env
    environment:
      - ENVROLE=dev

  curl-loop:
    image: curl-loop
    container_name: curl-loop
    hostname: curl-loop

where you could see mentioned before microservice’s port 4000 (such is quite usual for GraphQL services) mapped to host’s 8080 (mostly for GraphiQL UI usage purposes on host). And also defined 2 hostnames as rust-example and curl-loop for ease of use.

To start overall podman-compose configuration — such command is used:

$ podman-compose -f image/podman-compose.yml up -d

and if everything worked as expected — you should see 2 running containers after:

$ podman container ls
CONTAINER ID  IMAGE                                     COMMAND            CREATED         STATUS             PORTS                   NAMES
d3ca8b920f43  localhost/curl-loop:latest                tail -f /dev/null  17 minutes ago  Up 17 minutes ago                          curl-loop
3d3143f25170  localhost/rust-example-multi-arch:latest  /service           17 minutes ago  Up 17 minutes ago  0.0.0.0:8080->4000/tcp  rust-example

Testing time

Testing one query via podman and curl-loop container, while invoking curl utility inside it to hit rust-example container on internal port 4000:

$ podman exec -it curl-loop curl -d '{"query": "{howdy}"}' rust-example:4000
{"data":{"howdy":"partner"}}

and then testing another query:

$ podman exec -it curl-loop curl -d '{"query": "{details}"}' rust-example:4000
{"data":{"details":"Your requested details"}}

Voila, tests are working!

And to tear down all podman-compose configuration — you could use such:

$ podman-compose -f image/podman-compose.yml down

Tackling another architecture

At this moment satisfied with all working fine for x86_64 case — I thought naively that it will just magically work for aarch64 (in my case this is Armbian system based on Ubuntu 22.04 and running kernel 6.1.x). So I thought by exporting/importing images from/to Podman — I’ll be able to run all described solution as well fast.

And so I did for image export:

$ podman save -o ~/rust-example-multi-arch.tgz localhost/rust-example-multi-arch:latest

and for image import on aarc64 system:

$ podman load -i ~/rust-example-multi-arch.tgz

and checking failed to start microservice’s container logs — such error was observed:

{"msg":"exec container process `/service`: Exec format error","level":"error","time":"2023-02-13T15:27:01.203968Z"}

So what should I do without setting up whole Rust/building toolchain on Armbian system itself? Then I’ve found an article on cross-compilation by Sylvain Kerkour and followed custom Dockerfile approach with slight modification to use –release option as part of command line in mine image/toolchain/Dockerfile.aarch64 file:

FROM rust:latest

RUN apt update && apt upgrade -y
RUN apt install -y g++-aarch64-linux-gnu libc6-dev-arm64-cross

RUN rustup target add aarch64-unknown-linux-gnu
RUN rustup toolchain install stable-aarch64-unknown-linux-gnu

WORKDIR /app

ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
    CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
    CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
    RUSTFLAGS='-C target-feature=+crt-static'

CMD ["cargo", "build", "--release", "--target", "aarch64-unknown-linux-gnu"]

Such I’ve built into my local toolchain-aarch64 image:

$ podman build . -t toolchain-aarch64 -f image/toolchain/Dockerfile.toolchain-aarch64

and then used that to build aarch64 specific image via:

$ podman run --rm -ti -v `pwd`:/app toolchain-aarch64

Let’s compare 2 built executables

This is for x86_64 case:

$ file target/x86_64-unknown-linux-gnu/release/service
target/x86_64-unknown-linux-gnu/release/service: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), static-pie linked, BuildID[sha1]=da805b1383547ac61d0aed8321ed27c41b652fc1, for GNU/Linux 3.2.0, with debug_info, not stripped

and this is for aarch64 case:

$ file target/aarch64-unknown-linux-gnu/release/service
target/aarch64-unknown-linux-gnu/release/service: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=80f6c3f9fbb6bdcf69a5731ed8407618ea7fad18, for GNU/Linux 3.7.0, with debug_info, not stripped

At this point it’s possible to build Docker image for aarch64 with needed specific executable but not to override previously prepared for x86_64 case — I’ll suffix image tag (this will require re-tag later to use unchanged podman-compose configuration) with ‘-aarch64’:

$ podman build -t rust-example-multi-arch-aarch64 -f image/Dockerfile.aarch64 .

After that your prepared images should look like such:

$ podman image ls -a | grep -P "Repo|rust-example-multi"
localhost/rust-example-multi-arch-aarch64   latest      c79370704caf  46 seconds ago  14.2 MB
localhost/rust-example-multi-arch           latest      7eb5a6189149  8 days ago      14.7 MB

Now doing image podman save on Ubuntu x86_64 system and podman load on Armbian aarch64 system:

$ podman save -o ~/rust-example-multi-arch-aarch64.tgz localhost/rust-example-multi-arch-aarch64:latest 
$ podman load -i ~/rust-example-multi-arch-aarch64.tgz

and then re-tag:

$ podman tag rust-example-multi-arch-aarch64 rust-example-multi-arch

and then could start in exact same way podman-compose configuration on my Armbian system.

Short conclusion and future thinking

Achieved result of working solution on 2 different architectures brought me that feeling of ‘write once and run everywhere’ as it was some time in the past with me and Java and JVMs.

Also seeing ease of GraphQL approach preparation — I think it deserves more to be explored next.