Open Policy Agent and Gatekeeper to control Kubernetes Cluster

Pod Security Policies are depreciated in Kubernetes 1.21 and will be removed in future versions of K8s (1.25)
https://github.com/kubernetes/kubernetes/pull/97171

This link is from the above notice and talks about the problems they are having with PSP https://docs.google.com/presentation/d/1Kv6BSBNyLCyglMbK7e6tVOaDYe89LV2aHL2Hlb-9HX8/edit#slide=id.g5d1d1d6ca2_1_0 (slide 10)

The alternative suggested is Open Policy Agent and its integration to K8s via Gatekeeper project https://github.com/open-policy-agent/gatekeeper

The earlier integration of OPA with the K8s cluster was through a sidecar. With Gatekeeper this is simplified with an operator and the Policy through same Rego syntax, but as CRD /Custom Resource Defenitions Templates Please read the docs and this https://cloud.google.com/kubernetes-engine/docs/how-to/pod-security-policies-with-gatekeeper for an overview

Here is an example of using the same to prevent accidental deletion of namespaces.

~ Note that by default Gatekeeper does not notify on DELETE request. You need to add this option during installation

Install Gatekeeper version (pls update to latest) with the option of DELETE event

kubectl create -f https://gist.github.com/alexcpn/e4b627faca3c578783d1547f389bd1c3/raw/1739ec26793bc0f5895c068e642a0cb9b3a9679d/gatekeeper.yaml

Now lets modify the example and create a Constraint template

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          properties:
            annotation:
              type: array
              items: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels
violation[{"msg": msg}] {
            operation := input.review.operation
            operation == "DELETE"
            annotation := input.review.object.metadata.annotations
            test := annotation["protected"] == "false"
            not test
            msg := sprintf("you must provide annotation \"protected=false\" for namespace operation=%v DEBUG INFO annotation=%v,test=%v", [operation,annotation,test])
         }

Apply this

kubectl apply -f https://gist.github.com/alexcpn/e4b627faca3c578783d1547f389bd1c3/raw/1739ec26793bc0f5895c068e642a0cb9b3a9679d/constrainttemplate.yaml

Now the Constraint

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-gk
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    annotations: ["protected"]

Apply this

kubectl apply -f https://gist.github.com/alexcpn/e4b627faca3c578783d1547f389bd1c3/raw/1739ec26793bc0f5895c068e642a0cb9b3a9679d/constraint.yaml

Now lets test

kubectl create  ns test12 && kubectl annotate ns test12 protected=true
namespace/test12 created
namespace/test12 annotated

Now let’s try to delete it

kubectl delete ns test12
Error from server ([ns-must-have-gk] you must provide annotation "protected=false" for namespace operation=DELETE DEBUG INFO annotation={"protected": "true"},test=false): admission webhook "validation.gatekeeper.sh" denied the request: [ns-must-have-gk] you must provide annotation "protected=false" for namespace operation=DELETE DEBUG INFO annotation={"protected": "true"},test=false

And now lets set protected to false and try to delete

kubectl annotate ns test12 protected=false --overwrite
kubectl delete ns test12
namespace "test12" deleted

Note Namespaces without this annotation are not affected

kubectl create ns test12
namespace/test12 created
kubectl delete ns test12
namespace "test12" deleted

That’s it ; Overall nice; but documentation is not that great and you will struggle a bit with the rego flow