Azure Policy for Kubernetes with AKS

A detailed guide on how to enforce azure policies within configured AKS clusters.

What is Azure Policy?

Azure Policy is a feature of the Azure Resource Management Platform which allows organizations to define and enforce standards across their environments. Azure Policy for Kubernetes is a Kubernetes add-on that works with Azure Policy. The add-on extends extends Gatekeeper v3, an admission controller webhook for Open Policy Agent (OPA). Once installed, the add-on does the following:

• Check with Azure Policy Service for policy assignments to the cluster
• Deploys policy definitions into the cluster as constraint template and constraint custom resources
• Reports auditing and compliance details back to the Azure Policy Service

Lets Begin

We will divide this into three steps :

• Provision AKS Cluster
• Enable Azure Policy for Containers
• Using Azure Policy with Kubernetes

Provision AKS Cluster

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -

Checkout below repository and perform the terraform init, plan and apply steps.

GitHub - alim10081989/azure-aks: Terraform code to create AKS cluster on Azure

Save the output for kubeconfig and set it as an environment variable KUBECONFIG

Refer official documentation https://learn.microsoft.com/en-us/azure/aks/learn/quick-kubernetes-deploy-terraform?tabs=azure-cli for more details

Enable Azure Policy for Containers

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

To begin, check to see if Microsoft.PolicyInsights is registered in your Azure Subscription:

az provider show — namespace Microsoft.PolicyInsights

If the provider is not present, you must register it:

az provider register — namespace Microsoft.PolicyInsights

Next, let’s return to the AKS Cluster and install the add-on. Change the resource group and clusterName accordingly. During setup of AKS cluster via terraform in above it’s already enabled so below command can be skipped.

az aks enable-addons — resource-group $resourceGroupName — name $clusterName — addons azure-policy

Verify the add-on has been installed

az aks show — resource-group $resourceGroupName — name $clusterName — query addonProfiles.azurepolicy

Output should resemble the following

{
 // …
 "enabled": true,
 // …
}

Within kube-system, two deployment object named azure-policy and azure-policy-webhook would be created with their corresponding pods running. Also a new namespace named gatekeeper-system would be created with deployment object named gatekeeper-audit and daemonset object named gatekeeper-controller.

gatekeeper-system namespace

azure policy pods

Note: The more workloads that need to be evaluated, the more resources will be consumed by the Azure Policy and Gatekeeper components in the cluster. Therefore you need to ensure that you have enough resources to begin with and can scale accordingly in order to avoid unpredictably failing audit and enforcement operations.

Using Azure Policy for Kubernetes

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Azure Policy for Kubernetes may be viewed and managed through the Azure Portal. Simply open Azure Policy and navigate to the Definitions tab in the Authoring section.

Azure Policy Definitions

Expand the Category filter and clear all the checked items except for Kubernetes. This will filter the view down to just Azure Polciy for Kubernetes Definitions and Initiatives. An Initiative is a collection of one or more Definitions applied to a particular scope, like a Resource Group

Kubernetes Definition & Initiative

Azure Policy for Kubernetes provides some pre-defined intiatives. Let’s open and apply one to our infrastructure. Find the “Kubernetes cluster pod security baseline standards for Linux-based workloads” and click it. It will help if you apply a filter where “Definition Type = Initiative.”

Kubernetes Policy Initiative

The selected Initiative is comprised of five Policy Definitions. Notice that one is titled, Kubernetes cluster should not allow privileged containers. Let’s assign this initiative to our Cluster’s Resource Group.

Click the Assign initiativebutton located at the top of the Initiative tab.

Assign Initiative

With the Portal’s Assign Initiative wizard under the Basics tab, use the Scope control with subscription and Cluster’s Resource Group selected.

Then, on the Parameters tab, uncheck the Only show parameters that need input or review checkbox. This will update the screen and you will be able to change the value of the Effect control. Change Effect from Audit to deny.

AKS Policy Initiative Parameters

click Review + create followed by Create. This will apply the Initiative.

Note: It can take a substantial amount of time for the results of Azure Policy to be applied to your cluster — perhaps as long as thirty minutes.

Once the Initiative is in place, lets test it out by doing the following:

First, let’s create a namespace to hold our work.

kubectl create namespace azurepolicytest
kubectl config set-context --current --namespace azurepolicytest

Next, let’s run an interactive bash Pod on the cluster in privileged mode:

kubectl run test-shell --rm -i --tty --privileged=true --image ubuntu -- bash

This previous command should fail due to our new Azure Policy for Kubernetes Initiative.

AKS Privileged Pods denied

Let create a sample deployment object with privileged set to false and check if it allows the pod to be created.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: aks-helloworld
spec:
  replicas: 2
  selector:
    matchLabels:
      app: aks-helloworld
  template:
    metadata:
      labels:
        app: aks-helloworld
    spec:
      containers:
      - name: aks-helloworld-one
        image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
        securityContext:
          privileged: false

Apply the yaml and the pod should be running bcoz it doesn’t go against the policy set .

Change the privileged from false to true and apply. This will throw below warnings within events .

AKS Pod Policy Warnings

Check with Azure Policy and it should give you the details of the resource been non-compliant.

Non-Compliant Kubernetes Cluster

As final step , delete the namespace and clean up the cluster with terraform destroy.

kubectl delete namespace azurepolicytest
kubectl config set-context --current --namespace default

Congratulations 🎊, You’ve just set up a working azure policy within AKS cluster provisioned using terraform and successfully implemented across pods . In next chapter we will look into how to create custom azure policy for AKS.

Found something useful? Hold down the 👏 to support and help others find this article. Thanks for reading and have a good day. Cheers!!