Integrating DockerSlim container minify step on Cloud Build

Whenever we are dealing with containers, the general recommendation is to make them leaner, smaller, faster, and secure. The smaller the container image is, the faster your application will spin up for the first time, and the faster it will scale.

But more on this topic, the industry weighs the benefits of having a smaller container, to be more secure as reduces the available attack surface.

On serverless container environments like Cloud Run, having a leaner container image helps to reduce cold starts.

Integrating DockerSlim container minify step on Cloud Build

As more users run their application on Cloud Run, we at REEA.net were interested to explore getting DockerSlim into our build pipeline to create smaller and secure containers.

I have provided valuable feedback to the docker-slim team to make sure the tool can be used as part of a Cloud Build job. Starting with the 1.34.4 release Cloud Build integration is supported as they enhanced the IPC communications.

What is DockerSlim?

docker-slim is a tool that provides a set of commands (build, xray, lint and others) to reduce the size and optimize your containers. It makes your containers better, smaller, and more secure by using various techniques.

You don’t need to do any changes to your Dockerfile. docker-slim will optimize and secure your containers by understanding your application and what it needs. It disposes of packages and files from the container image that your application does not need to serve the business logic. It’s reported that it will minify it by up to 30x, for compiled languages even more.

Optimizing images isn’t the only thing it can do though. It can also help you understand and author better container images It will throw away what you don’t need, reducing the attack surface of your container.

docker-slim has been used with Node.js, Python, Ruby, Java, Golang, Rust, Elixir, and PHP (some app types) running on Ubuntu, Debian, CentOS, Alpine, and even Distroless.

Using DockerSlim

If you are new to docker-slim check out:

Integrating DockerSlim on Cloud Build

We use Cloud Build to build a new fat container. Then we need to run a docker-slim step to minify the fat container. Once done, push the resulting slim container to the container/artifact registry.

Hence we need the following steps:

  • Build the fat-container-app using the docker build command (this is the standard functioning of Cloud Build)
  • Download and install the latest docker-slim binary to the running environment — run the docker-slim utility to do the minify job
  • In the end, push the new slimcontainer to the container registry

The basic docker-slim command to reduce a fat container is:

docker-slim build image-name

Here is the full Cloud Build YAML file, the command is on Line 19.

Cloud Build YAML file with docker-slim integration

Cloud Build integration needs two params for the docker-slim utility to work correctly in this environment:

--sensor-ipc-mode proxy --sensor-ipc-endpoint <DOCKER_HOST>

To get the IP of the Docker service that runs under Cloud Build there is this magic command-line snippet, this will return likely 172.17.0.1

$(docker network inspect bridge -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}' | cut -f1)

You will have the below output

As the job will work, you will get in your logs these steps. This is a simplified view for the article.

Step #0 builds your Dockerfile

BUILD
Starting Step #0
Step #0: Sending build context to Docker daemon 7.887MB
....
Step #0: Successfully built 852516901a87
Finished Step #0

Step #1 will do the docker-slim minify

Starting Step #1
...
Step #1: docker-slim: message='join the Gitter channel to ask questions or to share your feedback' info='https://gitter.im/docker-slim/community'
...
Step #1: cmd=build state=started
Step #1: cmd=build state=image.inspection.start
Step #1: cmd=build info=image id='sha256:852516901a8712ea71600a9158000ef361fa3ceb787678794ffe3ebf809d9e83' size.bytes='413396128' size.human='413 MB'
...
Step #1: cmd=build info=image.exposed_ports list='80,8080'
Step #1: cmd=build state=http.probe.starting message=WAIT FOR HTTP PROBE TO FINISH
tep #1: cmd=build prompt='waiting for the HTTP probe to finish'
Step #1: cmd=build state=http.probe.running
Step #1: cmd=build info=http.probe.call error='none' time='2022-03-03T12:38:03Z' status='200' method='GET' target='http://172.17.0.1:49155/' attempt='1'
Step #1: cmd=build state=http.probe.done
...
Step #1: cmd=build state=container.inspection.done
Step #1: cmd=build state=building message=building optimized image
Step #1: cmd=build state=completed
Step #1: cmd=build info=results by='5.21X' size.original='413 MB' size.optimized='79 MB' status='MINIFIED'
...
Step #1: cmd=build info=report file='slim.report.json'
...
Finished Step #1

Step #2 will do the push of the myimage.slim container.

Starting Step #2
Step #2: The push refers to repository [.../myimage.slim]
Step #2: 2790c1ba5777: Preparing
Step #2: 2790c1ba5777: Pushed
Finished Step #2

❗ Please notice in the end you have an image with the .slim suffix:

Important elements to be aware of

  1. The docker-slim utility needs to connect to the underlying Docker host service to be able to do its thing.
    Cloud Build doesn’t expose easily the docker service, the “docker” hostname is not set, the DOCKER_HOST environment variable is also not set. But we worked closely with the docker-slim maintainers to provide them feedback to detect the Cloud Build environment and now you don’t need to deal with figuring out the configuration setup.
  2. Starting with the docker-slim release 1.34.4 Cloud Build integration is supported as they enhanced the IPC communications, the older versions won’t work on Cloud Build.
  3. If you encounter failure with the below error message, it means docker-slim cannot connect to the host Docker service. If you follow the steps in this article this should not be the case, only if you use an older version of docker-slim, than documented here.
“docker-slim: failure" error="wait timeout"”

The probe

In order for docker-slim to create a functional container, it uses an HTTP probe to trigger the app, so that it can observe what files are referenced, as those files cannot be stripped from the container.

It’s very important to ensure the HTTP probe works with success as that’s necessary for docker-slim to create a bootable and working container.

One of the most important phases here is connectivity, often people mess with the PORT param, so make sure will have multiple tries until you get a working endpoint, usually port 8080 works fine for all the Cloud Run environments, but your case could be different.

You can manually set the port and the app to be started by using these flags:

--new-expose=8080 --http-probe-ports=8080 --http-probe-exec 'curl --connect-timeout=5 http://localhost:8080/customPathToProbe'

Only do this if the default option won’t yield the connection to your app.

❗ It’s essential to be aware, that the triggered HTTP service should touch as many steps as it can from your app. If you know that some application folders need to be included in the slim container, you can add them with.

--include-path=/var/www/modules/ 
--include-path=/usr/lib/apache2/

We won’t cover in this article how to debug, and make a fully functional slim container, you have lots of resources about this on their documentation page.

Find our more about the specific docker-slim command line flags and features documented on their GitHub page.

Using the minified container.

The generated container is tagged with .slim suffix.

Once the container is built, you need to deploy it. We are not documenting here the steps to deploy it, you already have that in your pipeline.

What we found out working great, is that we use the minified container as a base container for our app. And we change and rebuild the slim=base container only when the mixture of the components changes.

Deploying turned out to have big speed improvements. As the slim container is stripped down of layers, and it’s smaller, the build and deploy time have been reduced drastically.

Conclusions

We have described how to integrate DockerSlim into Cloud Build as a minify step.

The created slim container is smaller, it’s stripped of unnecessary things, more secure as reduces the available attack surface.

You could have observed a basic PHP app based on Apache container, it was minified by 5X.

Step #1: cmd=build info=results by='5.21X' size.original='413 MB' size.optimized='79 MB' status='MINIFIED'

We have mentioned the necessity to have a working probe to create a functional container, and that based on your environment and stack you might spend time customizing the probe command flags in order to make it execute.

It’s important to understand that you do not need to rebuild the slim container on every deployment, only when the stack and essential components change in your containers.

If you have strong verification, unit, and functional tests in place before pushing to production. You can have a two-legged deployment when you don’t change your stack and packages. We use this frequently on our daily builds.

  1. Build the slim container out of a Dockerfile_basefile.
  2. Use the slim container as starting Dockerfile for your app, for fast packaging and deployment.
FROM myimage.slim
COPY . /var/www/html

Our production builds are faster, as we use the slim container in the build step, and only package the app files on top of it. It’s interesting to see how a 2-minute build process is reduced to 15 seconds. As this step is only placing the files on top of the container and does no other compile, reduce, slim steps.

Wrap Up

In the meantime, if you want to check it out, here are some links:

Feel free to reach out to me on Twitter @martonkodok or read my previous posts on medium/@martonkodok

--

--

--

A collection of technical articles and blogs published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Recommended from Medium

Continuous Development with Spring boot and Kubernetes

Youtube bot to automatically comment &like a video & subscribe to a channel

IT Vendor Lock-in Has Changed

Which Programming Language?

Knowing the way ….. Industrial IoT solution implements on GCP — Part I

Day 15 of #100DaysOfCode Challenge

Chat Button with GetButton.io

How to search and replace a Text String in a Word DOCX in C# .Net Framework

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Márton Kodok

Márton Kodok

Speaker at conferences, a Google Developer Expert top user on Stackoverflow, software architect at REEA.net, co-founder IT Mures, life-long learner, mentor

More from Medium

Secure Google Cloud SQL Instances using Private IP: Gotchas & troubleshooting

Modeling one-to-many relation in Firestore, Bigtable, Spanner

Serverless APIs made simple on GCP with Goblet backed by Cloud Functions and Cloud Run

GCP — Self-serve timed access to GCP resources using Cloud Identity and Slack — Part 1