Emulating ARM architecture in OCI Cloud Shell

Photo by Quang Nguyen Vinh: https://www.pexels.com/photo/shell-on-sea-beach-6872201/

What’s OCI Cloud Shell?

OCI Cloud Shell is so easy-to-use and powerful, it’s easy to take it for granted! It’s available in your OCI Console by clicking on the Cloud Shell icon (in the upper right of the screen):

Opening OCI Cloud Shell (upper-right of OCI Console)

OCI Cloud Shell comes pre-loaded with a lot of the tools and utilities that are needed to manage and interact with an OCI environment, including:

• OCI CLI
• Docker
• kubectl
• git
• terraform
• python
• … many more!

See the OCI Cloud Shell documentation for more information about what’s baked into this great resource.

The challenge with multi-architectural testing

It can be desirable (correction, necessary) to emulate different CPU architectures in OCI Cloud Shell, namely the ARM architecture. By default, OCI Cloud Shell provides an x86_64 compute environment that you can use to interact with your OCI account and other resources. Confirm this by running uname -p:

cloudshell:~ (us-sanjose-1)$ uname -p
x86_64
cloudshell:~ (us-sanjose-1)$

There are times when it might be needed to emulate the ARM architecture, like when you need to build a multi-architectural container image (or a container image for arm specifically) via Cloud Shell. Because Cloud Shell is so tightly integrated into OCI, it’s a great spot for creating one-off container images and uploading them to OCI Container Registry. Since it’s an x86_64 architecture, emulation is needed to simulate and build ARM container images (or perform other arm-based computational activities).

NOTE: I’m not advocating using Cloud Shell for building production container images and the like. This is more useful for experimentation purposes. Production workloads should be automated using OCI DevOps! Again, back to our one-off, experimentation use-case, let’s look at how ARM can be emulated in OCI Cloud Shell…

For a simple example, let’s look at a basic Dockerfile that will fail in Cloud Shell. Follow along in your Cloud Shell instance:

cloudshell:~ (us-sanjose-1)$ mkdir multi_arch_test
cloudshell:~ (us-sanjose-1)$ cd multi_arch_test
cloudshell:multi_arch_test (us-sanjose-1)$ nano Dockerfile

Paste the following contents into the file:

FROM container-registry.oracle.com/os/oraclelinux:8-slim
RUN microdnf update
CMD echo "Hello world!"

Type CTRL+x, type y, then press enter (to save the file as Dockerfile).

Now go ahead and try building the container:

docker build --pull --platform arm64 -f Dockerfile -t test:v1.0.0 .

You’ll notice that it fails. You should see something like the following:

cloudshell:multi_arch_test (us-sanjose-1)$ docker build --pull --platform arm64 -f Dockerfile -t test:v1.0.0 .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM container-registry.oracle.com/os/oraclelinux:8-slim
Trying to pull repository container-registry.oracle.com/os/oraclelinux ... 
8-slim: Pulling from container-registry.oracle.com/os/oraclelinux
Digest: sha256:abc123...
Status: Image is up to date for container-registry.oracle.com/os/oraclelinux:8-slim
 ---> fe659acbc8d5
Step 2/3 : RUN microdnf update
 ---> Running in 9ec24c4a4bbc
exec /bin/sh: exec format error
The command '/bin/sh -c microdnf update' returned a non-zero code: 1
cloudshell:multi_arch_test (us-sanjose-1)$

What’s happening here is we’re trying to run an aarch64 (aka arm64) image (and associated binaries) on an x86_64 (aka amd64) platform. It won’t work (out of the box, anyway).

Emulation to the rescue!

Our Cloud Shell session is in an x86_64 (aka amd64) environment, but we need to build/deploy for the aarch64 (aka arm64) architecture. Thankfully there are solutions to getting around the problem we’ve encountered above. Solutions such as qemu-user-static and qus let us emulate different hardware architectures. Let’s use qemu-user-static, by running the following in OCI Cloud Shell:

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

Now try building the sample container again:

cloudshell:multi_arch_test (us-sanjose-1)$ docker build --pull --platform arm64 -f Dockerfile -t test:v1.0.0 .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM container-registry.oracle.com/os/oraclelinux:8-slim
Trying to pull repository container-registry.oracle.com/os/oraclelinux ... 
8-slim: Pulling from container-registry.oracle.com/os/oraclelinux
Digest: sha256:abc098932...
Status: Image is up to date for container-registry.oracle.com/os/oraclelinux:8-slim
 ---> fe659acbc8d5
Step 2/3 : RUN microdnf update
 ---> Running in zyx345
Downloading metadata...
Downloading metadata...
Package                                                  Repository           Size
Upgrading:                                                                        
 oraclelinux-release-8:8.6-1.0.6.el8.aarch64             ol8_baseos_latest 84.1 kB
   replacing oraclelinux-release-8:8.6-1.0.5.el8.aarch64                          
Transaction Summary:
 Installing:        0 packages
 Reinstalling:      0 packages
 Upgrading:         1 packages
 Obsoleting:        0 packages
 Removing:          0 packages
 Downgrading:       0 packages
Downloading packages...
Running transaction test...
Updating: oraclelinux-release;8:8.6-1.0.6.el8;aarch64;ol8_baseos_latest
Cleanup: oraclelinux-release;8:8.6-1.0.5.el8;aarch64;installed
Complete.
Removing intermediate container abc123...
 ---> bbc9c6bdfb14
Step 3/3 : CMD echo "Hello world!"
 ---> Running in xyz789...
Removing intermediate container abc456...
 ---> 1dea891c90b9
Successfully built abc789...
Successfully tagged test:v1.0.0
cloudshell:multi_arch_test (us-sanjose-1)$

It works! I will warn you that the above container build is slow… that’s just how it is when emulating other platforms (often times there will be a performance hit), but it does work.

Now go ahead and run the container (why not?):

docker run -it --rm test:v1.0.0

You should see something like the following:

cloudshell:multi_arch_test (us-sanjose-1)$ docker run -it --rm test:v1.0.0
Hello world!
cloudshell:multi_arch_test (us-sanjose-1)$

If you find that you want a clean OCI Cloud Shell session without emulation enabled, simply click on View(upper left side of the screen) and then Restart:

Restarting OCI Cloud Shell

Remember that Cloud Shell sessions are ephemeral, with only your home directory being retained between uses/sessions. This means that when you start a new Cloud Shell session, you’ll need to enable emulation again. It’s no big deal, just something to remember.

Wrapping it up

OCI Cloud Shell is super powerful, coming preloaded with many tools that make working with OCI super easy. Being able to emulate ARM from within Cloud Shell opens up lots of possibilities that can make life much easier, especially when working with and supporting multi-architectural (x86_64 and aarch64 instance) environments.

Let’s talk about it! Join us on the developer Public Slack channel here.