Managed Identities on AKS with AzureIdentity CRDs

Published in

Synechron

4 min readFeb 12, 2020

Managed Identities are a great way to improve the security of our applications.

Instead of supplying a user and secret using config files or environment variables — which could be compromised — we use what we might call a Bourne Identity model.

The Bourne Identity, Universal Pictures (2002)

Our application, a.k.a. Jason Bourne, wakes up with amnesia. A friendly fisherman (Kubernetes) gives it the reminder it needs — to talk to the cloud provider’s metadata service. Here, its identity will be revealed.

Managed Identity on IaaS

Let’s use Azure as an example.

When we provision a VM, we can either choose, for simplicity, to let the platform provide our instance with a System Managed Identity, or we can choose to create and supply our own User Assigned Managed Identity.

*Creating a System Managed Identity for an Azure VM*

Then, just like with GCP and AWS, we can ask the host’s Metadata Service for a token. We use the non-routable IP 169.254.169.254 like so:

$ curl -X GET -H Metadata:true "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2019-06-01&resource=https://management.azure.com" | jq{
  "access_token": "<token lives here>",
  "expires_in": "28799",
  "expires_on": "1581402976",
  "ext_expires_in": "28799",
  "not_before": "1581373876",
  "resource": "https://management.azure.com",
  "token_type": "Bearer"
}

We can then use this token to authenticate with the Cloud platform’s APIs — and we never had a password.

Managed Identity on Kubernetes

On Kubernetes, we can’t generally choose the node that’s running our service, so we can’t just hit the Metadata Service directly.

Instead, Microsoft provide an elegant solution for Azure Kubernetes Service (AKS) by the way of two Custom Resource Definitions, hosted in the Azure/aad-pod-identity project on Github:

The CRDs are:

AzureIdentity
AzureIdentityBinding

First, we can install these by doing:

$ kubectl apply -f https://raw.githubusercontent.com/Azure/aad-pod-identity/master/deploy/infra/deployment-rbac.yaml

Note: drop ‘-rbac’ if the cluster isn’t RBAC-enabled

Representing a Managed Identity in Kubernetes

Let’s create a User Assigned Managed Identity.

There’s less configuration to do if we create the identity in the Resource Group that contains our AKS nodes. We can find this from the Portal, or using az as follows:

$ az aks list | jq -c '.[] | .nodeResourceGroup'"MC_redcluster-aks-rg_redcluster_westeurope"

With the Resource Group name in hand (represented by ${node_rg_name}), we can create an Managed Identity (replacing the ${name} and ${location} placeholders with our chosen name and location):

$ az identity create \
--name ${name} \
--resource-group ${node_rg_name} \
--location ${location}

Using the AzureIdentity CRD, we define a representation of our Managed Identity inside our cluster:

apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentity
metadata:
  name: ${name}
spec:
  type: 0
  ResourceID: /subscriptions/${subscription_id}/resourceGroups/${node_rg_name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/${name}
  ClientID: ${identity_client_id}

where the placeholders should be replaced as follows:

${name} — the name we’d like to give to the K8s object representing our Managed Identity
${subscription_id} — the Subscription ID in which our Managed Identity lives
${node_rg_name} — the Resource Group in which the Managed Identity was created (using the example above, the Resource Group containing our Azure Kubernetes nodes, typically starting with MC_)
${identity_client_id} — the Client ID of our Managed Identity (it will also have an “Object ID” — also called the “Principal ID”)

Mapping our Identity to our Pods

With our identity in place in our Kubernetes cluster, we can use an AzureIdentityBinding to define a mapping between our AzureIdentity and our chosen Pods:

apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentityBinding
metadata:
  name: ${name}-to-${selectorValue}
spec:
  AzureIdentity: ${name}
  Selector: ${selectorValue}

Here, the placeholders should be replaced with:

${name} — as above, the name of the AzureIdentity object defining our Managed Identity
${selectorValue} — the value of the aadpodidbinding label that we’ve applied to our Pods (keep reading)

The Selector here works a little bit differently to normal Kubernetes selectors.

The value of the Selector declared in the binding has to be present as as the value of a label called aadpodidbinding:

apiVersion: extensions/v1beta1
kind: Deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      run: my-app
  template:
    metadata:
      labels:
        ##### The label HAS to be called 'aadpodidbinding'
        aadpodidbinding: ${selectorValue}
        run: my-app
    spec:
      containers:
      ...

In creating this AzureIdentityBinding, we grant matching Pods the permission to assume the given AzureIdentity.

We can then grant our identity the permissions it needs — via familiar IAM Role Assignments.

Using our Managed Identity in an initContainer

Let’s imagine that we want our Pod to initialise itself with some data from Blob Storage when it is scheduled. It’s a great job for a KubernetesinitContainer.

All we need to provide is the Managed Identity ID with a suitable IAM role (e.g. Storage Blob Data Reader).

initContainers:
  command:
    - /bin/sh
    - -c
  - args:
      - azcopy login --identity --identity-client-id ${client_id}
        && azcopy copy https://${storage_account}.blob.core.windows.net/mybucket/* /var/data
  image: datenbetrieb/azcopy
  name: data-init
  volumeMounts: #### <-- our `container`, in the same pod, also mounts this volume
  - mountPath: /var/data
    mountPropagation: None
    name: data_init