Securing Containers With Google’s Container Optimized OS & Distroless Container Images

Computing infrastructure has many layers. For many modern containerized cloud workloads, you’ll need to consider both the virtual machine image used to run your Kubernetes worker nodes, and also the container image used as a base for your containerized applications.

Google has a solution for both. Container-Optimized OS is an operating system maintained by Google specifically designed for running Docker containers on Compute Engine VMs. “Distroless” Docker images are maintained by google, which contain only the application and its runtime dependencies. Combined, you’ll have a powerful and secure baseline for your cloud compute infrastructure.

Container-Optimized OS

If you run a Google Kubernetes Engine cluster, you’ll realize that the default node image is now Container-Optimized OS with containerd (cos_containerd). Google’s documentation states that it provides the following features and benefits:

• Run containers out of the box
• Smaller attack surface
• Locked-down by default
• Automatic Updates

It’s a Chromium based OS that delivers on these promises. Its minimal OS footprint means that it is trimmed down to only the essential requirements to run containers efficiently. It optimizes container runtime efficiency while reducing the attack surface by not including unnecessary tools and packages.

It delivers on its “Locked-down by default” promise by having an immutable root filesystem, stateless configuration, a security-hardened kernel, security-centric defaults, hardened default firewall, advanced instance access support. Its built by Google, adhering to their continuous vulnerability scanning and response and undergoes a thorough testing process.

And last but not least, nodes on Google Kubernetes Engine running Container-Optimized OS supports node auto-upgrades. These nodes can be automatically scheduled for upgrades, which are rolled out gracefully. Ensuring maximum uptime and security at no additional effort.

Distroless Docker Images

The design focus of optimization and security seen by Google’s Container-Optimized OS holds true for “Distroless” images, a Google Container Tools project.

“Distroless” images contain only your application and its runtime dependencies. They do not contain package managers, shells or any other programs you would expect to find in a standard Linux distribution.

Containerized applications should be designed to be stateless and immutable. If that is true, then there is little to no value in keeping a package manager, shell, and many other standard programs and utilities installed on the container. In most cases, all those utilities are doing is increase the attack surface of your container!

By removing unnecessary packages, Distroless images greatly reduce security risks while also improving operational efficiency.

Let’s take a look at a container security scan results for a nodejs app. First, we’ll look at a nodejs app directly on the node:16 docker image. Then we’ll compare it with the same nodejs app, but with an additional docker build stage to copy it to the distroless node image.

The original Dockerfile:

FROM node:16 AS build-env
COPY . /app
WORKDIR /app
RUN npm ci 
CMD [“app.js”]

We can then build and scan this image for vulnerabilities using Docker and Snyk.

docker build . -t node-example
snyk container test node-example

Yikes. The scan results summary does not look good.

Tested 412 dependencies for known issues, found 392 issues.

So lets fix this. Let’s add a second Docker build stage where we copy the built nodejs app to a distroless base image.

FROM node:16 AS build-env
COPY . /app
WORKDIR /app
RUN npm ci
FROM gcr.io/distroless/nodejs:16
COPY --from=build-env /app /app
WORKDIR /app
CMD ["app.js"]

Then once again build and scan. These results look MUCH better.

Tested 9 dependencies for known issues, found 10 issues.

Distroless Security Benefits

• Reduced attack surface
• No more executing interactive shells
• Verifiable with cosign

Distroless Operational Benefits

• Grealy reduced image size. `gcr.io/distroless/static-debian11` is less than 2% the size of the standard `debian` container image. This can greatly reduce times for rapid testing during CI/CD pipelines.
• Less time wasted on container vulnerability scanning false positives. Your security team will thank you.

Conclusion

By running distroless containers on Google’s Container-Optimized OS, you’ll be benefiting from their many features. For modern cloud native applications, these security and operational features are essential.