Integrate Open Policy Agent with IBM Multicloud Manager policy framework for Kubernetes resource admission control

Overview

Open Policy Agent (OPA) is a generic policy engine to help you make decisions based on the policy you defined using a lightweight programming language call Rego.

IBM Multicloud Manager policy framework provides a desired state-based management approach for informing and enforcing on the policy compliance of a set of clusters managed by Multicloud Manager.

In this article, I am going to show you how to combine the power of both to achieve the purpose of enforcing Kubernetes resource admission control with OPA policies in a flexible way.

Ingredients

• IBM Multicloud Manager 3.2.0 or later
• Open Policy Agent 0.12.0
• Kubernetes command-line interface (kubectl)

Installing OPA on IBM Multicloud Manager

OPA provides well written documentation on how to install OPA on Kubernetes and integrate with Kubernetes admission control. In this article, I will provide the following instructions from the OPA documentation with minor modifications for IBM Multicloud Manager.

To make it easier, you can access the source code from my GitHub repository.

ycao56/mcm-opa

Complete the following steps to install OPA on Kubernetes and integrate with Kubernetes admission control:

1. Clone the Github repository:

Run the following command to clone the `ycao56/mcm-opa.git` repository:

git clone git@github.com:ycao56/mcm-opa.git

2. Configure kubectl to point to the cluster managed by IBM Multicloud Manager.

The steps to configure kubectl could vary depending on each Kubernetes cluster. If you have an IBM Cloud Private cluster, you can configure it by completing the following steps:

1. Log in to your cluster.
2. Select the user icon, then click Configure client.
3. Copy and paste the configuration information into your command line, and press Enter.

3. Create a namespace for OPA.

Run the following command:

kubectl create namespace opa

We will install OPA related components into the opa namespace.

4. Install OPA and configure the admission controller webhook.

Run the following commands:

cd mcm-opa
kubectl apply -f yaml/opa -n opa

When you run the commands it will perform the following tasks:

1. Install OPA as a deployment.
2. Create a secret.
3. Create role and role binding.
4. Register admission controller webhook.

Applying a IBM Multicloud Manager policy to enforce OPA policy

After OPA is installed and configured, it is time to create an OPA policy. OPA deployments use a sidecar to load an OPA policy from the ConfigMap. Instead of creating a ConfigMap directly, we will use IBM Multicloud Manager policy framework to enforce the creation to ensure the ConfigMap that contains OPA policy always exist on the cluster that IBM Multicloud Manager is managing.

In the GitHub repository, I created a IBM Multicloud Manager policy with an OPA policy embedded. To apply it on your cluster, run following command:

kubectl apply -f yaml/mcm/policy-opa.yaml -n kube-system

Policy explanation

Here is the magic:

IBM Multicloud Manager policy framework provides the capability to create any Kubernetes object on managed clusters by using object-template. You can define one or more Kubernetes object inside object-templates. By setting the complianceType parameter to musthave and remediationAction to enforce, IBM Multicloud Manager policy framework will make sure the cluster it applies to has the defined objects created.

In this example, a ConfigMap object is embedded which contains the OPA policy. This policy denies any pod, whose image URL does not start with hooli.com, to be created in the opa namespace.

Testing the results

Now it is time to see if the policy actually works.

First, let’s check if the policy has been loaded into OPA.

Run following command to verify the policy is loaded:

kubectl get configmap -n opa no-pod -oyaml

You might receive the result:

apiVersion: v1
data:
  no_pod.rego: |-
    package kubernetes.admission
    deny[msg] {
      input.request.kind.kind == “Pod”
      input.request.namespace == “opa”
      image := input.request.object.spec.containers[_].image
      not startswith(image, “hooli.com”)
      msg := sprintf(“image fails to come from trusted registry: %v”, [image])
 }
kind: ConfigMap
metadata:
  annotations:
    openpolicyagent.org/policy-status: ‘{“status”:”ok”}’
    creationTimestamp: “2019–07–01T17:17:23Z”
  name: no-pod
  namespace: opa
  resourceVersion: “8735582”
  selfLink: /api/v1/namespaces/opa/configmaps/no-pod
  uid: 17915dd5–9c24–11e9–9cd6–005056a061f1

The annotation `openpolicyagent.org/policy-status: ‘{“status”:”ok”}’` represents the that the policy has successfully loaded into OPA.

Now let’s try to create a pod that violates the policy. Run the following command:

kubectl apply -f yaml/pod.yaml -n opa

You might receive the following response because the request was rejected with following errors:

Error from server (image fails to come from trusted registry: nginx, image fails to come from trusted registry: mysql): error when creating “yaml/pod.yaml”: admission webhook “validating-webhook.openpolicyagent.org” denied the request: image fails to come from trusted registry: nginx, image fails to come from trusted registry: mysql

Let’s try to create it in a different namespace.

Run the following command:

kubectl apply -f yaml/pod.yaml -n default

When your pod is created, you might receive the following message:

pod/myapp created

That’s it! You have successfully integrated OPA with IBM Multicloud Manager policy framework to do admission control on Kubernetes with your own OPA policy.

Conclusion

It this article, we walked through the steps required to integrate OPA with IBM Multicloud Manager policy framework. We enabled the admission control using OPA engine on clusters that IBM Multicloud Manager manages by defining a IBM Multicloud Manager policy to enforce the creation of OPA policy. Finally, we tested the OPA policy by creating a pod that violates the policy rule.

References

1. Guides: Kubernetes Admission Control — https://www.openpolicyagent.org/docs/latest/guides-kubernetes-admission-control/
2. Multicloud Manager policy framework — https://www.ibm.com/support/knowledgecenter/SSBS6K_3.2.0/mcm/compliance/policy_overview.html