Achieving Multi-Cloud Security: Integrating Sensitive Data in .NET 6 Applications on AKS with secrets-store-csi-driver

In the rapidly evolving landscape of cloud-native applications, ensuring the security of sensitive data has become paramount. With the synergy of .NET 6 and Kubernetes, we’ve found a robust solution to this challenge through the secrets-store-csi-driver. Not only does this driver enable seamless integration of secrets into our .NET 6 applications on Azure Kubernetes Service (AKS), but it also extends its reach to Google Cloud Platform (GCP) and Amazon Web Services (AWS). Join us as we explore the power of secrets-store-csi-driver, a unified approach to securing secrets across multiple clouds.

Introduction

In an era where data breaches can be detrimental, safeguarding sensitive information is crucial. Our team recently embarked on a journey to fortify our .NET 6 applications running on AKS with a modern security solution. The secrets-store-csi-driver emerged as a game-changer, offering a standardized method to incorporate secrets while maintaining a multi-cloud architecture.

The Secrets-Store-CSI-Driver

Secrets Store CSI Driver for Kubernetes secrets — Integrates secrets stores with Kubernetes via a Container Storage Interface (CSI) volume.

The Secrets Store CSI Driver secrets-store.csi.k8s.io allows Kubernetes to mount multiple secrets, keys, and certs stored in enterprise-grade external secrets stores into their pods as a volume. Once the Volume is attached, the data in it is mounted into the container’s file system.

The secrets-store-csi-driver acts as a bridge between our applications and secret stores, allowing us to securely access sensitive information as environment variables. One of the standout features of this driver is its adaptability across cloud providers, enabling consistent integration regardless of the chosen platform.

Features

• Multiple external secrets store providers
• Pod portability with the SecretProviderClass CustomResourceDefinition
• Mounts secrets/keys/certs to pod using a CSI Inline volume
• Mount multiple secrets store objects as a single volume
• Linux and Windows containers

Unified Configuration Across Clouds

What truly sets secrets-store-csi-driver apart is its ability to handle secrets uniformly across various cloud platforms. Whether we’re deploying on Azure, GCP, or AWS, the configuration remains consistent. This harmonized approach not only simplifies our development process but also enhances security by reducing configuration errors.

Installation and Setup

Implementing secrets-store-csi-driver for our .NET 6 applications proved to be surprisingly straightforward. By following a few steps, we seamlessly integrated this driver into our AKS clusters. Furthermore, the same configuration translated seamlessly to our GCP and AWS deployments, streamlining our multi-cloud strategy. We chose the Helm while installing it.

helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver --namespace kube-system

Leveraging the Multi-Cloud Advantage

The multi-cloud approach provides several benefits, including redundancy, avoiding vendor lock-in, and cost optimization. With secrets-store-csi-driver, we can securely tap into these advantages by deploying the same codebase across different cloud providers without compromising security.

The configuration for Azure provider seems like this:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-kvname-user-msi
spec:
  provider: azure
  secretObjects:                                 # [OPTIONAL] SecretObject defines the desired state of synced K8s secret objects
  - secretName: redis-variables
    type: Opaque
    data:
    - objectName: redis-connectionstring # name of the mounted content to sync. this could be the object name or object alias
      key: redis-connectionstring
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true" # true since using managed identity
    userAssignedIdentityID: $KV_IDENTITY_RESOURCE_ID
    keyvaultName: $KEY_VAULT_NAME
    cloudName: ""
    objects:  |
      array:
        - |
          objectName: redis-connectionstring
          objectType: secret    # object types: secret, key, or cert
          objectVersion: ""     # default to latest if empty 
    tenantId: $TENANT_ID

The KV_IDENTITY_RESOURCE_ID is your identity that has access to Key-Vault. Here are some tutorials about how to create and get the value of this variable:

https://learn.microsoft.com/en-us/azure/aks/csi-secrets-store-identity-access

Quick Guide to the Secret Store CSI driver for Azure Key Vault on AKS

The TENANT_ID is the Azure AD tenant Id where your identities live. It is better to pass them through the library variable while creating your manifest for each environment accordingly.

This SecretProviderClass will create a secret from the values of Key-Vault defined, and let you mount it for deployment.

An example of our deployment file:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: $DEPLOYMENTNAME
spec:
  replicas: $REPLICAS
  selector:
    matchLabels:
      app: $DEPLOYMENTNAME
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: $STRATEGY_ROLLING_UPDATE_MAX_SURGE
      maxUnavailable: $STRATEGY_ROLLING_UPDATE_MAX_UNAVAILABLE
  template:
    metadata:
      labels:
        app: $DEPLOYMENTNAME
    spec:
      volumes:
        - name: secrets-store01-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "azure-kvname-user-msi"
      containers:
        - name: $DEPLOYMENTNAME
          image: $REGISTRY/$IMAGE:$VERSION
          ports:
            - containerPort: 80
          #########
          # Probes#
          #########
          livenessProbe:
            httpGet:
              path: /liveness
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 15
            successThreshold: 1
            failureThreshold: 3
            timeoutSeconds: 2
          readinessProbe:
            httpGet:
              path: /readiness
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 30
            successThreshold: 1
            failureThreshold: 3
            timeoutSeconds: 2
          volumeMounts:
            - name: secrets-store01-inline
              mountPath: "/mnt/secrets-store"
              readOnly: true
          env:
            - name: "Redis__ConnectionString"
              valueFrom:
                secretKeyRef:
                  name: redis-variables
                  key: redis-connectionstring
            - name: ASPNETCORE_ENVIRONMENT
              value: $ASPNETCORE_ENVIRONMENT  
          resources:
            limits:
              cpu: $RESOURCES_LIMITS_CPU
              memory: $RESOURCES_LIMITS_MEMORY
            requests:
              cpu: $RESOURCES_REQUESTS_CPU
              memory: $RESOURCES_REQUESTS_MEMORY

This configuration seems complicated. But in simple terms, after SecretProviderClass creates the secret, we get the value from that secret and pass it to our application as an environment variable. The trick is using a double underscore to set and override a value in the appsettings.json file. Here is the explanation:

The : separator doesn't work with environment variable hierarchical keys on all platforms. __, the double underscore, is:

Supported by all platforms. For example, the : separator is not supported by Bash, but __ is.

Automatically replaced by a :

Known Limitations

• When the secret/key is updated in the external secrets store after the initial pod deployment, the updated secret is not automatically reflected in the pod mount or the Kubernetes secret.
• When the SecretProviderClass is updated after the pod was initially created.
• Adding/deleting objects and updating keys in existing secretObjects ones doesn’t result in the update of Kubernetes secrets.

How to fetch the latest content without Auto rotation the feature enabled?

1. If the SecretProviderClass has secretObjects defined, then delete the Kubernetes secret.
2. Restart the application pod.

Best Practices for Multi-Cloud Secrets Management

While the driver simplifies the integration process, best practices for secrets management remain critical. Regular secret rotation, secure access policies, and data encryption are still paramount to maintaining a robust security posture across clouds.

Testing and Deployment Across Clouds

Ensuring the stability of our applications is even more critical. By thoroughly testing the applications in our multi-cloud environment, we verified that secrets were being accessed correctly and securely. Our deployment strategies now encompass Azure, GCP, and AWS with consistent security measures intact.

Conclusion

Secret management and security in this era of dynamic cloud deployments are crucial. Our journey with secrets-store-csi-driver has not only elevated the security of our .NET 6 applications but has also unlocked the true potential of multi-cloud deployments. With a unified approach to configuration and management, this driver has demonstrated its prowess in streamlining security across Azure, GCP, and AWS. As we navigate the complexities of modern application deployments, secrets-store-csi-driver stands as a beacon of multi-cloud security and consistency.

Thank you for joining us on this journey into the world of secure multi-cloud deployments with secrets-store-csi-driver. But this is just the beginning. We’re committed to sharing more advanced tips, real-world scenarios, and expert strategies in upcoming articles.

As cloud technology continues to evolve, we’ll be your guides, helping you navigate the ever-changing landscape. Whether you’re a seasoned pro or new to the scene, stay tuned for more insights that will empower your applications and enhance your cloud security.

Keep an eye out for our future articles. Your questions and feedback are always welcome as we delve deeper into the intricacies of modern application deployment and multi-cloud security. Until then, happy coding and secure cloud journeys ahead!

#devops #devsecops #secretmanagement #cloud #security