Hands on with Kubernetes Pod Security Admission

Photo by Eric Prouzet on Unsplash

Kubernetes v1.22 provides an alpha release for the successor of Pod Security Policy (PSP) (which is scheduled for deprecation in v1.25). This new enhancement is called Pod Security Admission (PSA). I’ve taken an initial look at PSA and will cover what you need to know about how it works and how you can start playing with it. This will not be a deep dive into the architecture or reasoning behind the implementation. If you want to learn more about that you can find details in the Kubernetes Enhancement Proposal (KEP).

One obvious difference between PSP and PSA is that PSA only appears to operate as a validating admission controller and doesn’t appear to support mutating resources like PSP.

NOTE: Kubernetes features that are in alpha aren’t typically available for use by managed Kubernetes services like AKS, EKS, GKE for stability reasons. Please check your providers documentation

Enabling Pod Security Admission

To enable Pod Security Admission you will need a v1.22 Kubernetes cluster with the following feature flag enabled --feature-gates="...,PodSecurity=true". When testing new alpha features, I like to use a tool called KinD which allows you to quickly spin up a Kubernetes cluster locally. Below is the cluster configuration I used to spin up a v1.22 Kubernetes cluster with this feature enabled. Save the YAML below to a file named kind-config.yaml. Special callout to Jim Angel who provided this KinD configuration.

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
 PodSecurity: true
nodes:
- role: control-plane
- role: worker

You can then start a new cluster using the following command:

$ kind create cluster — image=kindest/node:v1.22.0@sha256:b8bda84bb3a190e6e028b1760d277454a72267a5454b57db34437c34a588d047 — config kind-config.yaml`
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.22.0) 🖼
 ✓ Preparing nodes 📦 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
 ✓ Joining worker nodes 🚜
Set kubectl context to "kind-kind"
You can now use your cluster with:
kubectl cluster-info --context kind-kind
Thanks for using kind! 😊

Configuring Policy

Now that you have a cluster up and running the next thing you will need to do is configure policies. Before we cover how to configure policies we need to understand Pod Security Standards. These standards define three different policies that range from permissive to restrictive. These three policies are as follows:

• Privileged — open and unrestricted
• Baseline — Covers most common known privilege escalations and provides an easier onboarding
• Restricted — Highly restricted, covering best practices. May cause compatibility issues

Each of these policies define which fields are restricted within a Pod specification and the allowed values. Some of the fields restricted by these policies include:

• spec.containers[*].ports
• spec.volumes[*].hostPath
• spec.securityContext
• spec.containers[*].securityContext

Now that we know what policies are available they can be applied in two different ways either via namespace labels or an AdmissionConfiguration resource. Using namespace labels allows for granulator per namespace policy selection whereas using AdmissionConfiguration allows cluster-level defaulting along with exemptions.

Policy Modes

Policies are applied using modes. Here is a list of modes:

• enforce — Any Pods that violate the policy will be rejected
• audit — Pods with violations will be allowed and an audit annotation will be added
• warn — Pods that violate the policy will be allowed and a warning message will be sent back to the user.

In addition to modes you can also pin the policy to a specific version for example v1.22. Pinning to a specific version allows the behavior to remain consistent as policy changes happen over Kubernetes releases.

Applying Policy to a namespace

Policies are applied to a namespace via labels. These labels are as follows:

• REQUIRED: pod-security.kubernetes.io/<MODE>: <LEVEL>
• OPTIONAL: pod-security.kubernetes.io/<MODE>-version: <VERSION> (defaults to latest)

The following command warns the baseline Pod Security Standard pinned to v1.22 of the policy on the test-ns namespace. Be

kubectl label --overwrite ns test-ns \
  pod-security.kubernetes.io/warn=baseline \
  pod-security.kubernetes.io/warn-version=v1.22

As you can imagine, enforcing policy on a namespace with existing workloads could be disruptive. You can run the following command on a cluster to evaluate existing workloads against the policy and determine which workloads will need to be modified so they won’t violate the policy.

$ kubectl label --dry-run=server --overwrite ns --all \
    pod-security.kubernetes.io/enforce=baseline
Warning: kuard: privileged
namespace/default labeled
namespace/kube-node-lease labeled
namespace/kube-public labeled
Warning: kube-proxy-vxjwb: host namespaces, hostPath volumes, privileged
Warning: kube-proxy-zxqzz: host namespaces, hostPath volumes, privileged
Warning: kube-apiserver-kind-control-plane: host namespaces, hostPath volumes
Warning: etcd-kind-control-plane: host namespaces, hostPath volumes
Warning: kube-controller-manager-kind-control-plane: host namespaces, hostPath volumes
Warning: kindnet-cl5ln: non-default capabilities, host namespaces, hostPath volumes
Warning: kube-scheduler-kind-control-plane: host namespaces, hostPath volumes
Warning: kindnet-6ptww: non-default capabilities, host namespaces, hostPath volumes
namespace/kube-system labeled
namespace/local-path-storage labeled

Another really neat trick is that you can apply more than a single mode to a specific namespace. This is helpful to enforce at one policy standard and warn and audit at another. In the following example Pods that violate the baseline policy won’t be allowed and Pods that violate the restricted policy will warn and audit.

apiVersion: v1
kind: Namespace
metadata:
  name: test-ns
  labels:
    pod-security.kubernetes.io/enforce: baseline
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

To demonstrate the behavior of the configuration above lets create a NGINX Pod with the following manifest.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - image: nginx
      name: nginx
      ports:
        - containerPort: 80

In the output below we can see that we receive a warning as the Pod violates the restricted policy and lists how to remediate. The Pod does not violate the baseline policy which is being enforced and hence is created.

$ kubectl apply -f pod.yaml
Warning: would violate "latest" version of "restricted" PodSecurity profile: allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
pod/nginx created
laevenso@feu [(⎈ |kind-kind:test-ns)]  in ~
$ kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          6s

If you have Kubernetes audit logging enabled you can see an audit message for restricted policy violations as configured by the namespace labels. Audit logging isn’t currently supported natively in Kind however Jim Angel as provided the following Kind config that enables audit logging. Thanks Jim!

{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"808ca159-914c-43fa-b4c8-dee5cb2fc440","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default/pods?fieldManager=kubectl-create","verb":"create","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["172.18.0.1"],"userAgent":"kubectl/v1.22.0 (darwin/amd64) kubernetes/c2b5237","objectRef":{"resource":"pods","namespace":"default","name":"nginx","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":201},"requestReceivedTimestamp":"2021-08-21T03:30:26.605589Z","stageTimestamp":"2021-08-21T03:30:26.627123Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","pod-security.kubernetes.io/audit":"allowPrivilegeEscalation != false (container \"nginx\" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container \"nginx\" must set securityContext.capabilities.drop=[\"ALL\"]), runAsNonRoot != true (pod or container \"nginx\" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container \"nginx\" must set securityContext.seccompProfile.type to \"RuntimeDefault\" or \"Localhost\")"}}

Applying cluster-wide policy

In addition to applying labels to namespaces to configure policy you can also configure cluster-wide policies and exemptions using the AdmissionConfiguration resource. Using this resource, policy definitions are applied cluster-wide by default and any policy that is applied via namespace labels will take precedence. There is no runtime configurable API for this resource and a cluster administrator would need to specify a path to the file below via the --admission-control-config-fileflag on the API server. In this resource we are enforcing the baseline policy and warning and auditing the restricted policy. We are also making the kube-system namespace exempt from this policy.

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: DefaultPodSecurity
  configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1alpha1
    kind: PodSecurityConfiguration
    defaults:
      enforce: "baseline"
      enforce-version: "latest"
      audit: "restricted"
      audit-version: "latest"
      warn: "restricted"
      warn-version: "latest"
    exemptions:
      usernames: []
      runtimeClassNames: []
      namespaces: [kube-system]

Migrating from PSP to PSA

I haven’t yet taken an in-depth look at migration from PSP to PSA however here are the docs if you’re interested.

Summary

Having spent a considerable amount of time working with PSP, the successor PSA appears to be a much simpler implementation and achieves the same result. Pod Security Standards make it much easier for users to apply security best practices without going into painstaking detail to understand each and every security related field in a Pod specification. Namespace labelling also makes it easy to apply policy and choose the mode the policy operates. I suspect that Pod Security Policy will be easier to use and hence will be widely adopted. Overall this is a good thing for Kubernetes security. Finally, it’s worth mentioning again that this feature is still in alpha (at the time of writing) and is subject to change. Always consult the Kubernetes official documentation for the most up to date docs.