Securing Kubernetes with Open Policy Agent (OPA)

7 min readMar 28, 2023

Introduction:

Kubernetes is a powerful tool that helps us manage and deploy our software in an efficient and reliable way. However, as our systems become more complex, it can be challenging to ensure that they are secure and compliant with rules and regulations. That’s where Open Policy Agent (OPA) comes in. In this article, we’ll explore how OPA can be used with Kubernetes to secure our systems and enforce policy.

What is OPA?

Before diving into OPA in Kubernetes, it’s worth reviewing what OPA is and how it works. OPA is a policy engine that works by evaluating policies at runtime. It provides a centralized way to manage policies that can be applied across multiple applications and environments. OPA uses a policy language called Rego, which lets developers write policies in a way that’s both human-readable and machine-executable. Once the policies are written, they’re stored in a central policy repository that can be accessed by the OPA engine. You can read fundamental Knowledge of OPA with a given example in the below link.

Open Policy Agent: A Powerful Policy Engine for Cloud-Native Environments

In today’s fast-paced digital world, organizations need to enforce policies that ensure the security, reliability, and…

medium.com

How OPA Works with Kubernetes:

OPA has native integrations with Kubernetes, which makes it easy to deploy and use OPA in a Kubernetes environment. There are several ways that OPA can be integrated with Kubernetes:

OPA with Gatekeeper

One of the most popular ways is to use a tool called OPA Gatekeeper. OPA Gatekeeper is a Kubernetes validating admission controller that enforces policies on Kubernetes objects as they are created or modified. When a user or process attempts to create or modify a Kubernetes object. OPA Gatekeeper intercepts the request and checks it against the policies defined in OPA. If the request violates a policy, it is rejected.

OPA with Service Mesh:

Another way to use OPA with Kubernetes is by using a service mesh like Istio or Envoy. Service meshes help us manage the traffic between our different services, and we can use OPA to ensure that this traffic is secure and follows our policies. For example, we can use OPA to block traffic to a service that does not have the correct authorization or to encrypt traffic between services.

In this article we will discuss and see how Gatekeeper can be installed and use step by step .

OPA Gatekeeper Overview

As we mentioned earlier OPA Gatekeeper is a validating admission controller which will be called through Kubernetes webhook. Validating admission controllers are used to validate whether a Kubernetes resource is permitted with a set of rules or policies before it is created or updated in the actual system.

Installation of OPA Gatekeeper

Cluster Admin permission is required to install the OPA gatekeeper.So need to create role and role-binding to gave access OPA gatekeeper to all the resources in Kubernetes.

There are many ways to install the OPA gatekeeper, we can use Helm command as follows. A basic Helm chart exists in charts/gatekeeper. If you have Helm installed, you can deploy via the following instructions for Helm v3

$ helm repo add gatekeeper https://open-policy-agent.github.io/gatekeeper/charts --force-update
"gatekeeper" has been added to your repositories

Those who don’t know what helm is, Helm is a package manager for Kubernetes. It helps you install, manage, and upgrade applications on a Kubernetes cluster. Think of it like a tool that helps you simplify the deployment and management of Kubernetes applications.

After running above command you can see the deployment results like below.

helm install gatekeeper/gatekeeper --name-template=gatekeeper --namespace gatekeeper-system --create-namespace
NAME: gatekeeper
LAST DEPLOYED: Wed Mar 22 09:21:54 2023
NAMESPACE: gatekeeper-system
STATUS: deployed
REVISION: 1
TEST SUITE: None

Components of the OPA gatekeeper are installed in the gatekeeper-system namespace. By executing the following command, information such as installed services and pads can be viewed.

$ kubectl -n gatekeeper-system get all                                                                       
NAME                                                 READY   STATUS    RESTARTS      AGE
pod/gatekeeper-audit-549bcc6775-z74gm                1/1     Running   1 (92s ago)   100s
pod/gatekeeper-controller-manager-7785cd6b4c-j64zv   1/1     Running   0             100s
pod/gatekeeper-controller-manager-7785cd6b4c-mcfv5   1/1     Running   0             100s
pod/gatekeeper-controller-manager-7785cd6b4c-xr9kt   1/1     Running   0             100s

NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/gatekeeper-webhook-service   ClusterIP   10.96.120.252   <none>        443/TCP   100s

NAME                                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/gatekeeper-audit                1/1     1            1           100s
deployment.apps/gatekeeper-controller-manager   3/3     3            3           100s

NAME                                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/gatekeeper-audit-549bcc6775                1         1         1       100s
replicaset.apps/gatekeeper-controller-manager-7785cd6b4c   3         3         3       100s

OPA Gatekeeper Constraint Template

A ConstraintTemplate resource is essentially a template for generating Constraint resources, which are the actual Kubernetes objects that enforce the policies defined in the ConstraintTemplate. The ConstraintTemplate resource defines the Rego code that will be used to evaluate Kubernetes objects against the set of policies defined in the ConstraintTemplate.

Let’s create a ConstraintTemplate that block all deployment requests that are missing our private registry private.example.com

$ cat <<EOF | kubectl apply -f -
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8sallowedrepos
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedRepos
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          properties:
            repos:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sallowedrepos

        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          satisfied := [good | repo = input.parameters.repos[_] ; good = contains(container.image, repo)]
          not any(satisfied)
          msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
        }

        violation[{"msg": msg}] {
          container := input.review.object.spec.initContainers[_]
          satisfied := [good | repo = input.parameters.repos[_] ; good = contains(container.image, repo)]
          not any(satisfied)
          msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
        }
EOF

$ constrainttemplate.templates.gatekeeper.sh/k8sallowedrepos created

We used bellow rego code above to create, If you see closely we are checking valid image repo names of a regular & initContainer containers in a k8s deployment request payload.

package k8sallowedrepos

violation[{"msg": msg}] {
  container := input.review.object.spec.containers[_]
  satisfied := [good | repo = input.parameters.repos[_] ; good = contains(container.image, repo)]
  not any(satisfied)
  msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
}

violation[{"msg": msg}] {
  container := input.review.object.spec.initContainers[_]
  satisfied := [good | repo = input.parameters.repos[_] ; good = contains(container.image, repo)]
  not any(satisfied)
  msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
}

OPA Gatekeeper Constraint

Constraints can be thought of as templates for Kubernetes objects that enforce a specific set of policies. Each constraint is composed of two main parts: the spec and the parameters. The spec defines the set of policies that you want to enforce on Kubernetes objects, while the parameters section allows you to pass in variables that can be used in the spec.

We will create bellow constraints (K8sAllowedRepos) to inform Gatekeeper that the admin wants above ConstraintTemplate to be enforced, and Gatekeeper will apply and make sure to only allow valid image repos to be deployed in Kubernetes system.

$ cat <<EOF | kubectl apply -f -
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
  name: allow-only-private-registry
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    repos:
      - "private.example.com"
EOF

$ K8sAllowedRepos.constraints.gatekeeper.sh/allow-only-private-registry created

NOTE: We’ll want to change private.example.com to match the name of our own Image registry host. Also, we can have more than one registry in the list.

Now Let’s see if constraint works as expected. First, let’s deploy a pod that use private.example.com repository for image which satisfies the constraints.

$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: allowed-image-pod
spec:
  containers:
    - name: allowed-image-pod
      image: private.example.com/sample-api:tagname
      resources:
        limits:
          cpu: "100m"
          memory: "30Mi"
EOF          

pod/allowed-image-pod created

The pod is deployed so Gatekeeper didn’t denied our deployment,

Now let’s create another deployment which will not use our allowed image.

$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: deny-image-pod
spec:
  containers:
    - name: deny-image-pod
      image: docker.io/library/nginx:latest
      resources:
        limits:
          cpu: "100m"
          memory: "30Mi"
EOF          

Error from server (Forbidden): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [allow-only-private-registry] container <deny-image-pod> has an invalid image repo <docker.io/library/nginx:latest>, allowed repos are ["private.example.com"]

Cool 😎, So Gatekeeper denied our requests to deploy not allowed images.

This is how we can create many policies with the help of Gatekeeper to keep our Kubernetes system secured.

OPA Gatekeeper Policy Library

The OPA Gatekeeper Policy Library is a collection of pre-built policies that you can use with OPA Gatekeeper. The library includes policies for enforcing security best practices, compliance requirements, resource quotas, and more. You can use these policies as-is or customize them to fit your specific needs.

The library includes a set of directories, each containing a set of policies focused on a particular area such as security or compliance. Each policy is defined as a Rego file, and the directory also includes a README file with more information about the policies.

Conclusion:

Kubernetes is a powerful tool that helps us manage and deploy our software, but it can be challenging to ensure that our systems are secure and compliant. Open Policy Agent (OPA) provides a powerful solution for enforcing policies and securing our systems. By using OPA with Kubernetes, we can simplify policy enforcement, increase security, and ensure compliance with regulations.