Best Practices for Securely Accessing Azure Key Vault Secrets in OpenShift Applications

4 min readOct 11, 2023

Introduction

In today’s cloud-native landscape, securing sensitive data, such as API keys, database passwords, and encryption keys, is of paramount importance. As organizations increasingly adopt containerized applications on platforms like OpenShift, the need for a secure and efficient way to manage secrets becomes crucial.

In this blog post, we’ll explore how to sync secrets from Azure Key Vault to an OpenShift cluster using the External Secrets Operator. This powerful tool simplifies secret management and helps ensure that your applications have secure and easy access to the credentials they need.

Architecture Design

Prerequisites

Before diving into the tutorial, here are the prerequisites you’ll need to follow along:

Azure Account: You’ll need an active Azure account to create and manage an Azure Key Vault.
OpenShift Cluster: Access to an OpenShift cluster where you have administrative privileges or permissions to create resources.
External Secrets Operator: Ensure that the External Secrets Operator is installed on your OpenShift cluster. You can install it via OperatorHub or follow the operator’s documentation.

High Level Designs

High-Level Design of Implementing External Secret Operator (ESO) with Azure Key Vault on OpenShift Container Platform (OCP):

Set up Azure Key Vault:
- Create an Azure Key Vault instance and configure access policies.
- Generate or import the required secrets into Azure Key Vault
Configure ESO:
- Deploy ESO as a Kubernetes operator on the OCP cluster.
- Configure ESO to connect with Azure Key Vault using appropriate authentication mechanisms.
Define Secret Objects:
- Create Secret objects in OCP representing the secrets required by applications.
- Specify the reference to the corresponding secrets stored in Azure Key Vault.
Secret Injection:
- Create Secret objects in OCP representing the secrets required by applications.
- Specify the reference to the corresponding secrets stored in Azure Key Vault.
Security Considerations:
- Enable encryption in transit and at rest for secrets stored in Azure Key Vault.
- Implement proper access controls and permissions in Azure Key Vault.
- Monitor and audit access to secrets in Azure Key Vault for security compliance
Testing and Deployment:
- Validate the integration between ESO, Azure Key Vault, and OCP.
- Test the secret injection and retrieval process for applications.
- Deploy the solution to production, ensuring proper backup and disaster recovery mechanisms

Section 1: Setting up Azure Key Vault

Creating an Azure Key Vault-

Azure Key Vault is a secure and scalable cloud service for safeguarding sensitive data. Here’s how to create one:

Log in to the Azure Portal.
Click on “Create a resource” and search for “Key Vault.”
Follow the wizard to configure your Key Vault, specifying details like the name, resource group, and region.

Configuring Access Policies-

To allow the External Secrets Operator to access the Key Vault, configure access policies:

In the Key Vault menu, navigate to “Access policies.”
Add a new access policy, granting permissions for “Get” and “List” secrets to the operator’s managed identity.
Save the policy settings.

Section 2: Installing External Secrets Operator

The External Secrets Operator simplifies secret management. You can install it from OperatorHub:

Open the OpenShift Console.
Go to the “OperatorHub” section.
Search for “External Secrets” and install the operator.

Section 3: Creating an SecretStore

SecretStore is a namespace scoped resource that specifies how to connect to an external secrets manager. Create the SecretStore resource in the eso-azkv-demo namespace replacing $TENANT_ID and $VAULT_NAME with the values obtained previously as shown below:

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: azure-backend
  namespace: eso-azkv-demo
spec:
  provider:
    azurekv:
      authSecretRef:
        clientId:
          key: ClientID
          name: azure-secret-XX
        clientSecret:
          key: ClientSecret
          name: azure-secret-spn
      authType: ServicePrincipal
      environmentType: PublicCloud
      tenantId: 99g2e344-g33h-8677-4444-erfd1ee8f6r9
      vaultUrl: 'https://KEYVAULT-DEMO-OCPv4.vault.azure.net'

Section 4: Creating an External Secret

Now, let’s create an External Secret to sync with Azure Key Vault. This External Secret references the combined secret stored in Azure Key Vault. Apply the External Secret to create a Kubernetes Secret:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: example-secret-01
  namespace: eso-azkv-demo
spec:
  data:
    - remoteRef:
        conversionStrategy: Default
        decodingStrategy: None
        key: example-secret-01
      secretKey: secret
  refreshInterval: 5m
  secretStoreRef:
    kind: SecretStore
    name: azure-backend
  target:
    creationPolicy: Owner
    deletionPolicy: Retain
    name: example-secret-01

Confirm the secret was successfully synced by viewing the Conditions on the ExernalSecret created previously.

Section 5: Mounting and Using Secrets in Your Application

In your application deployment configuration, access and parse the secret:

containers:
  - name: my-app
    image: my-app-image
    env:
      - name: APP_USERNAME
        valueFrom:
          secretKeyRef:
            name: my-app-secret
            key: app-credentials

Use the APP_USERNAME environment variable in your application code.

Section 6: Testing and Troubleshooting

Test your setup and troubleshoot common issues. Ensure that your service account has the necessary permissions to access the secret.

Conclusion:

Syncing secrets from Azure Key Vault to OpenShift using the External Secrets Operator streamlines secret management in Kubernetes/OpenShift environments. It enhances security, simplifies deployments, and ensures that sensitive information is handled safely.

Additional Resources

For further learning and reference, check out these resources: