Open in app

Sign in

Write

Sign in

Sail Sharp, 9 tips to optimize and secure your .NET containers for Kubernetes

Mathieu Benoit
Google Cloud - Community
Published in
13 min readApr 1, 2023

Updated on May 31st, 2024 — Move base container images from alpine to chiseled (distroless).

Updated on November 15th, 2023 — .NET 8 is out + Native AOT support.

Updated on July 8th, 2023 — All the source code is now in mathieu-benoit/sail-sharp.

In February 2021, I got this opportunity to deliver this talk Sail Sharp, .NET Core & Kubernetes for the .NET Meetup in Quebec city (it was in French). I illustrated the best practices to prepare any .NET applications for Kubernetes. I was using the cartservice app (in .NET) from the very popular Online Boutique sample apps.

Since then, I have been one of the top contributors to the Online Boutique repository. I contributed to the golang, python, dotnet, java and nodejs apps. This repo was my playground. I learned a lot. Some of my contributions, among others, were about optimizing and securing the container images for all these apps.

Here is the high-level timeline of my contributions related to the cartservice app:

As a side note, I started my career with the .NET Framework version 3.0 (only on Windows at that time) back in 2006! Since then, I have been amazed about the evolution of the .NET ecosystem. And to be honest all of these contributions gave me a reason to stay up-to-date and have a lot of fun while learning more about containers and Kubernetes! :)

Wow! Quite a ride, isn’t it?

Today, in this blog post, I will highlight 8 tips to optimize and secure your .NET containers based on what I have learned with all of that:

  1. Use the multi-stage build approach
  2. Reduce the size of the bundled application
  3. Reduce the size of the base container images
  4. Use immutable base container images
  5. Update dependencies with Dependabot or Renovate
  6. Reduce the size of your final container with .dockerignore
  7. Secure unprivileged/non-root container
  8. Protect read-only container filesystem
  9. Accelerate startup time with compiled native code

tl,dr

If you want to see the final Dockerfile and the Deployment manifest to deploy a secure and optimized .NET application in Kubernetes, feel free to directly jump to the end of this blog post.

Disclaimer

Whereas most of the concepts could be applicable to Windows containers, this blog post is only covering Linux containers.

Also, I don’t take into account the generic multi-platform approach like illustrated here: Improving multi-platform container support — .NET Blog (microsoft.com). Something for future update I guess!

Create a minimal ASP.NET app

Create a folder where we will drop all the files needed for this blog post:

mkdir my-sample-app

Create a minimal and simple ASP.NET app that we will use for this blog post:

cat <<EOF > my-sample-app/Program.cs
using Microsoft.AspNetCore.Builder;
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello, World!");
app.Run();
EOF
cat <<EOF > my-sample-app/my-sample-app.csproj
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>
</Project>
EOF

1. Use the multi-stage build approach

Create our first Dockerfile with a multi-stage build. That’s not the final one, until then, please bear with me:

cat <<EOF > my-sample-app/Dockerfile
FROM mcr.microsoft.com/dotnet/sdk:8.0 as builder
WORKDIR /app
COPY my-sample-app.csproj .
RUN dotnet restore my-sample-app.csproj \
    --use-current-runtime
COPY . .
RUN dotnet publish my-sample-app.csproj \
    --use-current-runtime \
    -c release \
    -o /my-sample-app \
    --no-restore

FROM mcr.microsoft.com/dotnet/aspnet:8.0
WORKDIR /app
COPY --from=builder /my-sample-app .
ENTRYPOINT ["/app/my-sample-app"]
EOF

This Dockerfile uses multi-stage build, which optimizes the final size of the image by layering the build and leaving only required artifacts.

Let’s build this container image locally:

docker build \
    -t my-sample-app \
    my-sample-app/

We can see that the size of the container image is 220MB locally on disk.

You can now run this container:

docker run \
    -d \
    -p 8080:8080 \
    my-sample-app

Important to notice here, that now in .NET 8, the default port is not anymore 80 (privileged) but now on is 8080 (unprivileged), great to see security best practices applied by default here!

You can now test that this container is working successfully:

curl localhost:8080

Great, congrats!

2. Reduce the size of the bundled application

When using dotnet publish, we can use different features to optimize the size of the bundled application:

Update the Dockerfile with dotnet publish --self-contained true -r linux-x64 and dotnet restore -r linux-x64.

Update the Dockerfile with dotnet/runtime-deps:8.0 as the final base image.

We can see that the size of the container image is now 223MB on disk locally.

Should you use self-contained or framework-dependent publishing in Docker images?

Update the Dockerfile with dotnet publish -p:PublishTrimmed=True -p:TrimMode=full.

We can see that the size of the container image is now 223MB on disk locally.

Note: If your application doesn’t work with -p:TrimMode=full, you can use -p:TrimMode=partial instead.

Update the Dockerfile with dotnet publish -p:PublishSingleFile=true.

We can see that the size of the container image is now 216MB on disk locally.

3. Reduce the size of the base container images

To reduce the surface of attack or to avoid dealing with security vulnerabilities debt, using the smallest base image is a must.

You can find all the dotnet container images available here:

We can choose the alpine one: dotnet/runtime-deps:8.0-alpine3.19, if we rebuild the image we could see that now the size of the container image is down to 108MB.

We can also choose the chiseled one (i.e. distroless): dotnet/runtime-deps:8.0-noble-chiseled, if we rebuild the image we could see that now the size of the container image is down to 106MB.

Very impressive! Isn’t it?!

Below is the illustration of the sizes of the different base images:

REPOSITORY                              TAG                            IMAGE ID      CREATED      SIZE
mcr.microsoft.com/dotnet/runtime-deps   8.0-noble-chiseled             207757c4d1a4  2 days ago   14.3 MB
mcr.microsoft.com/dotnet/runtime-deps   8.0-cbl-mariner2.0-distroless  facd2de56680  2 days ago   24.9 MB
mcr.microsoft.com/dotnet/runtime-deps   8.0                            177964573070  2 days ago   124 MB
mcr.microsoft.com/dotnet/runtime-deps   8.0-alpine3.19                 2e67123bd84b  2 days ago   15.9 MB

Here, like you can see, I decided to take the smallest container image dotnet/runtime-deps:8.0-noble-chiseled: 14.3MB.

Note: the 8.0-cbl-mariner2.0-distroless tag is not yet listed here, but you can find it here.

Now, what is the difference between dotnet/runtime-deps:8.0-cbl-mariner2.0-distroless (24.9MB), dotnet/runtime-deps:8.0-noble-chiseled (14.3MB) and dotnet/runtime-deps:8.0-alpine3.19 (15.9MB)?

Good question, glad you asked!

-cbl-mariner2.0-distroless and -noble-chiseled are very attractive because they are bringing the concept of distroless. They are container images that do not contain the complete or full-blown OS with system utilities installed. You can read more about CBL-Mariner 2.0 here (not GA yet), and more about Chiseled Ubuntu Containers here (now GA and noble as LTS).

Note: Chainguard is also working on having their own distroless images for dotnet. Something to keep in mind too!

This blog post Image sizes miss the point explains really well why the distroless ones are very attractive:

To reduce debt, reduce image complexity not size.

By using a tool like syft, we could see that the distroless ones are less complex than the alpine one, with less dependencies, reducing the debt and surface of risks. See results below.

For mcr.microsoft.com/dotnet/runtime-deps:8.0-cbl-mariner2.0-distroless:

[10 packages]
NAME                         VERSION               TYPE
distroless-packages-minimal  0.1-4.cm2             rpm
filesystem                   1.1-20.cm2            rpm
glibc                        2.35-7.cm2            rpm
libgcc                       11.2.0-8.cm2          rpm
libstdc++                    11.2.0-8.cm2          rpm
mariner-release              2.0-62.cm2            rpm
openssl-libs                 1.1.1k-30.cm2         rpm
prebuilt-ca-certificates     2415459:2.0.0-17.cm2  rpm
tzdata                       2024a-1.cm2           rpm
zlib                         1.2.13-2.cm2          rpm

For mcr.microsoft.com/dotnet/runtime-deps:8.0-alpine3.19:

[17 packages]
NAME                    VERSION                TYPE
alpine-baselayout       3.4.3-r2               apk
alpine-baselayout-data  3.4.3-r2               apk
alpine-keys             2.4-r1                 apk
apk-tools               2.14.0-r5              apk
busybox                 1.36.1-r15             apk
busybox-binsh           1.36.1-r15             apk
ca-certificates-bundle  20240226-r0            apk
libc-utils              0.7.2-r5               apk
libcrypto3              3.1.5-r0               apk
libgcc                  13.2.1_git20231014-r0  apk
libssl3                 3.1.5-r0               apk
libstdc++               13.2.1_git20231014-r0  apk
musl                    1.2.4_git20230717-r4   apk
musl-utils              1.2.4_git20230717-r4   apk
scanelf                 1.3.7-r2               apk
ssl_client              1.36.1-r15             apk
zlib                    1.3.1-r0               apk

Very important note that this alpine container image contains busybox with wget included, which could help someone who made it into the running container to download malicious files, etc.

For mcr.microsoft.com/dotnet/runtime-deps:8.0-noble-chiseled:

[7 packages]
NAME             VERSION                TYPE
base-files       13ubuntu10             deb
ca-certificates  20240203               deb
libc6            2.39-0ubuntu8.1        deb
libgcc-s1        14-20240412-0ubuntu1   deb
libssl3t64       3.0.13-0ubuntu3.1      deb
libstdc++6       14-20240412-0ubuntu1   deb
zlib1g           1:1.3.dfsg-3.1ubuntu2  deb

Note: Another important point is the fact that alpine is based on musl, on the other hand, the distroless ones are based on glibc. This blog post: Why I Will Never Use Alpine Linux Ever Again highlights some known issues with alpine/musl. Good to keep in mind too.

Furthermore, and for your information, I gave trivy a try for these three container images, here is the summary of the scans:

  • For mcr.microsoft.com/dotnet/runtime-deps:8.0-alpine3.19:
Total: 6 (UNKNOWN: 0, LOW: 0, MEDIUM: 6, HIGH: 0, CRITICAL: 0)

┌───────────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┬────────────────────────────────────────────┐
│    Library    │ Vulnerability  │ Severity │ Status │ Installed Version │ Fixed Version │                   Title                    │
├───────────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┼────────────────────────────────────────────┤
│ busybox       │ CVE-2023-42363 │ MEDIUM   │ fixed  │ 1.36.1-r15        │ 1.36.1-r17    │ busybox: use-after-free in awk             │
│               │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2023-42363 │
│               ├────────────────┤          │        │                   ├───────────────┼────────────────────────────────────────────┤
│               │ CVE-2023-42366 │          │        │                   │ 1.36.1-r16    │ busybox: A heap-buffer-overflow            │
│               │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2023-42366 │
├───────────────┼────────────────┤          │        │                   ├───────────────┼────────────────────────────────────────────┤
│ busybox-binsh │ CVE-2023-42363 │          │        │                   │ 1.36.1-r17    │ busybox: use-after-free in awk             │
│               │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2023-42363 │
│               ├────────────────┤          │        │                   ├───────────────┼────────────────────────────────────────────┤
│               │ CVE-2023-42366 │          │        │                   │ 1.36.1-r16    │ busybox: A heap-buffer-overflow            │
│               │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2023-42366 │
├───────────────┼────────────────┤          │        │                   ├───────────────┼────────────────────────────────────────────┤
│ ssl_client    │ CVE-2023-42363 │          │        │                   │ 1.36.1-r17    │ busybox: use-after-free in awk             │
│               │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2023-42363 │
│               ├────────────────┤          │        │                   ├───────────────┼────────────────────────────────────────────┤
│               │ CVE-2023-42366 │          │        │                   │ 1.36.1-r16    │ busybox: A heap-buffer-overflow            │
│               │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2023-42366 │
└───────────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┴────────────────────────────────────────────┘
  • For mcr.microsoft.com/dotnet/runtime-deps:8.0-cbl-mariner2.0-distroless:
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
  • For mcr.microsoft.com/dotnet/runtime-deps:8.0-noble-chiseled:
Total: 4 (UNKNOWN: 0, LOW: 4, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

┌────────────┬────────────────┬──────────┬──────────┬───────────────────┬───────────────┬────────────────────────────────────────────────────────────┐
│  Library   │ Vulnerability  │ Severity │  Status  │ Installed Version │ Fixed Version │                           Title                            │
├────────────┼────────────────┼──────────┼──────────┼───────────────────┼───────────────┼────────────────────────────────────────────────────────────┤
│ libc6      │ CVE-2016-20013 │ LOW      │ affected │ 2.39-0ubuntu8.1   │               │ sha256crypt and sha512crypt through 0.6 allow attackers to │
│            │                │          │          │                   │               │ cause a denial of...                                       │
│            │                │          │          │                   │               │ https://avd.aquasec.com/nvd/cve-2016-20013                 │
├────────────┼────────────────┤          │          ├───────────────────┼───────────────┼────────────────────────────────────────────────────────────┤
│ libssl3t64 │ CVE-2024-2511  │          │          │ 3.0.13-0ubuntu3.1 │               │ openssl: Unbounded memory growth with session handling in  │
│            │                │          │          │                   │               │ TLSv1.3                                                    │
│            │                │          │          │                   │               │ https://avd.aquasec.com/nvd/cve-2024-2511                  │
│            ├────────────────┤          │          │                   ├───────────────┼────────────────────────────────────────────────────────────┤
│            │ CVE-2024-4603  │          │          │                   │               │ openssl: Excessive time spent checking DSA keys and        │
│            │                │          │          │                   │               │ parameters                                                 │
│            │                │          │          │                   │               │ https://avd.aquasec.com/nvd/cve-2024-4603                  │
│            ├────────────────┤          │          │                   ├───────────────┼────────────────────────────────────────────────────────────┤
│            │ CVE-2024-4741  │          │          │                   │               │ openssl: Use After Free with SSL_free_buffers              │
│            │                │          │          │                   │               │ https://avd.aquasec.com/nvd/cve-2024-4741                  │
└────────────┴────────────────┴──────────┴──────────┴───────────────────┴───────────────┴────────────────────────────────────────────────────────────┘

With all of that being said (size, number of packages and number of CVEs), I have decided to use the runtime-deps:8.0-noble-chiseled.

4. Use immutable base images

Use a specific tag or version for your base image, not latest is important for traceability. But a tag or version is mutable, which means that you can’t guarantee which content of the container you are using. Using a digest will guarantee that, a digest is immutable.

Update the Dockerfile with these two base images:

mcr.microsoft.com/dotnet/sdk:8.0.301-noble@sha256:daeec618239ba57630b19d572bbd55b4af66940fa564058355550fc93d86153f
mcr.microsoft.com/dotnet/runtime-deps:8.0.6-noble-chiseled@sha256:1745aa8e322b6965ea3ec8524b9f9eb128f146c804e32b1fc2272ed07ada4c8e

Note: it’s also highly encouraged that you store these two base images in your own private container registry and update your Dockerfile to point to them. You will guarantee their provenance, you will be able to scan them, etc.

5. Update dependencies with Dependabot or Renovate

An important aspect is to keep your dependencies up-to-date in order to fix CVEs, catch new features, etc. One way to help you with that, in an automated fashion, is to leverage tools like Renovate or Dependabot if you are using GitHub.

Here is an example of how you can configure Dependabot to keep your container base images as well as your .NET packages up-to-date:

cat <<EOF > .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "docker"
    directory: "/my-sample-app"
    schedule:
      interval: "daily"
  - package-ecosystem: "nuget"
    directory: "/my-sample-app"
    schedule:
      interval: "daily"
EOF

6. Reduce the size of your final container with .dockerignore

Use a .dockerignore file to ignore files that do not need to be added to the image. For examples, any bin, debug, obj, etc. folders that you may generate and need if you build and test your application locally.

Here is an example of how your .dockerignore could look like:

cat <<EOF > my-sample-app/.dockerignore
**/*.sh
**/*.bat
**/bin/
**/obj/
**/out/
Dockerfile*
EOF

7. Secure unprivileged/non-root container

For security purposes, always ensure that your images run as non-root by defining USER in your Dockerfile.

Since .NET 8, ASP.NET Core apps now listen on port 8080 by default. Before that and up until .NET 7, it was listening on port 80. The problem is that port 80 is a privileged port that requires root permission. For making the container unprivileged, even if it’s already by default the case, we will configure its port to 8080:

EXPOSE 8080
ENV ASPNETCORE_HTTP_PORTS=8080
USER 65532

Note: Until .NET 7, you may have seen something like this: ENV ASPNETCORE_HTTP_URLS=http://*:8080., instead of ENV ASPNETCORE_HTTP_PORTS=8080.

You can now run this container with -u 65532 on port 8080:

docker run \
    -d \
    -p 80:8080 \
    -u 65532 \
    my-sample-app
curl localhost:80

8. Protect read-only container filesystem

In .NET 7, to make the container in read-only mode on filesystem, DOTNET_EnableDiagnostics needed to be turned off. DOTNET_EnableDiagnostics is used for debugging, profiling, and other diagnostics.

ENV DOTNET_EnableDiagnostics=0

You can now run this container with --read-only:

docker run \
    -d \
    -p 80:8080 \
    -u 65532 \
    --read-only my-sample-app
curl localhost:80

This part is not anymore required with .NET 8.

9. Accelerate startup time with compiled native code

So a last step you can take from here, is optimizing the startup time of you app by compiling it with native code with a feature called: Native AOT.

Publishing your app as Native AOT produces an app that’s self-contained and that has been ahead-of-time (AOT) compiled to native code. Native AOT apps have faster startup time and smaller memory footprints.

I learn a lot from this resource too: The minimal API AOT compilation template (andrewlock.net).

To leverage this feature, you will need to remove the -p:PublishSingleFile=truetrue parameter in your Dockerfile otherwise you will have an error message:

PublishAot and PublishSingleFile cannot be specified at the same time

Then you will add these lines in your .csproj file:

    <PublishAot>true</PublishAot>
    <OptimizationPreference>Size</OptimizationPreference>
    <InvariantGlobalization>true</InvariantGlobalization>
    <StackTraceSupport>false</StackTraceSupport>

You will also need to use the nightly container images because these images supporting AOT are for now only provided in Public Preview.

What you can also do is using CreateSlimBuilder instead of CreateBuilder like explained here: Comparing WebApplication.CreateBuilder() to the new CreateSlimBuilder() method (andrewlock.net).

With all of that, when you will build your container, you will pass from 107MB to 42.4MB, impressive, isn’t it?!

You can find more information about the performance improvement with .NET 8 here: Performance Improvements in ASP.NET Core 8 — .NET Blog (microsoft.com).

That’s a wrap!

Congrats!

With these 9 tips illustrated throughout this blog post, we:

  • Reduced the size of the application bundled and the container itself
  • Reduced the surface of attack of the container image ( distroless was chosen)
  • Illustrated tips to improve the day-2 operations in order to keep our dependencies up-to-date
  • Made the container running as unprivilege/non-root in read-only on filesystem

Here is the final Program.cs:

using Microsoft.AspNetCore.Builder;
var builder = WebApplication.CreateSlimBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello, World!");
app.Run();

Here is the final .csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <PublishAot>true</PublishAot>
    <OptimizationPreference>Size</OptimizationPreference>
    <InvariantGlobalization>true</InvariantGlobalization>
    <StackTraceSupport>false</StackTraceSupport>
  </PropertyGroup>
</Project>

Here is the final Dockerfile:

FROM mcr.microsoft.com/dotnet/nightly/sdk:8.0.301-noble-aot@sha256:d0744082df452f2ba315ae610e07955a84ddd1c9bdf5c65ac4e068eaa8ae0764 as builder
WORKDIR /app
COPY my-sample-app.csproj .
RUN dotnet restore my-sample-app.csproj \
    -r linux-x64
COPY . .
RUN dotnet publish my-sample-app.csproj \
    -r linux-x64 \
    -c release \
    -o /my-sample-app \
    --no-restore \
    --self-contained true \
    -p:PublishTrimmed=true \
    -p:TrimMode=full

FROM mcr.microsoft.com/dotnet/nightly/runtime-deps:8.0.6-noble-chiseled-aot@sha256:b9d82f51f75362f22683e9432f23407280dbc1be5dcbbafda9cbfc251a01fb01
WORKDIR /app
COPY --from=builder /my-sample-app .
EXPOSE 8080
ENV ASPNETCORE_HTTP_PORTS=8080
USER 65532
ENTRYPOINT ["/app/my-sample-app"]

If you want to deploy this container image in a secure manner locally, here the associated command:

docker run \
    -d \
    -p 8080:8080 \
    --read-only \
    --cap-drop=ALL \
    --user=65532 \
    my-sample-app
curl localhost:8080

If you want to deploy this container image in a secure manner in Kubernetes, here is the associated Deployment resource:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-sample-app
  labels:
    app: my-sample-app
spec:
  selector:
    matchLabels:
      app: my-sample-app
  template:
    metadata:
      labels:
        app: my-sample-app
    spec:
      automountServiceAccountToken: false
      securityContext:
        fsGroup: 65532
        runAsGroup: 65532
        runAsNonRoot: true
        runAsUser: 65532
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: my-sample-app
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            readOnlyRootFilesystem: true
          image: ghcr.io/mathieu-benoit/my-sample-app:latest
          ports:
            - containerPort: 8080
      nodeSelector:
        kubernetes.io/os: linux

You can find all the source code here: mathieu-benoit/sail-sharp.

You can also find the source code of the cartservice gRPC-based app in the OnlineBoutique repository.

Finally, if you want to learn more about how to enforce such unprivileged capabilities for your containers, I invite you to read my other blog post: Improve your Kubernetes security posture, with the Pod Security Admission (PSA).

You are now ready to Sail Sharp! Hope you enjoyed that one! Cheers!

Originally posted at mathieu-benoit.github.io.

Kubernetes
Dotnet
Containers
Security
Distroless

--

--

Mathieu Benoit
Google Cloud - Community

Written by Mathieu Benoit

203 Followers
Writer for

Customer Success Engineer at Humanitec | CNCF Ambassador | GDE Cloud

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams