Kubernetes Security — Use Kyverno Policy Reporter to fix Kyverno deployment

In this story I will show how to deploy Kyverno components in a local Kubernetes cluster, and use policy-reporter to spot violations in Kyverno related deployments when using the Pod Security Standards (Restricted) profile.

Then I will apply the necessary changes to fix the violations, verifying the fixes make policy-reporter happy.

Create a local Kubernetes cluster

Let’s create a simple Kubernetes cluster with Kind by running the script below:

kind create cluster --image "kindest/node:v1.23.3" --config - <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
  - role: control-plane
  - role: control-plane
  - role: worker
  - role: worker
  - role: worker
EOF

This will give us a 3 master and 3 worker nodes cluster.

Deploy Kyverno and Kyverno policies

We can now deploy Kyverno and Kyverno policies with Helm:

helm upgrade --install --wait --timeout 15m --atomic \
  --namespace kyverno --create-namespace \
  --repo https://kyverno.github.io/kyverno kyverno kyverno \
  --values - <<EOF
replicaCount: 3
config:
  resourceFilters: []
EOF
helm upgrade --install --wait --timeout 15m --atomic \
  --namespace kyverno --create-namespace \
  --repo https://kyverno.github.io/kyverno kyverno-policies \
  kyverno-policies --values - <<EOF
podSecurityStandard: restricted
validationFailureAction: audit
EOF

This will bring Kyverno policy engine and Pod Security Standards (Restricted) policies in our local cluster.

Deploy Policy reporter with UI

Again, we are going to use Helm to deploy policy-reporter:

helm upgrade --install --wait --timeout 15m --atomic \
  --namespace kyverno --create-namespace \
  --repo https://kyverno.github.io/policy-reporter \
  policy-reporter policy-reporter --values - <<EOF
kyvernoPlugin:
  enabled: true
ui:
  enabled: true
  plugins:
    kyverno: true
EOF

Once deployed, we should be able to access the policy-reporter dashboard:

# port forward to the reporter ui service
kubectl port-forward -n kyverno svc/policy-reporter-ui 8080:8080

The dashboard should be browsable at http://localhost:8080 and Kyverno components related reports are at http://localhost:8080/policy-reports/Kyverno?namespaces=kyverno.

Fix seccomp profile violations

From the dashboard above, we can observe that Kyverno deployments violate the restrict-seccomp-strict policy.

In order to fix the restrict-seccomp-strict violations, we need to explicitly set the seccompProfile.type stanza of the securityContext:

helm upgrade --install --wait --timeout 15m --atomic \
  --namespace kyverno --create-namespace \
  --repo https://kyverno.github.io/kyverno kyverno kyverno \
  --values - <<EOF
replicaCount: 3
config:
  resourceFilters: []
podSecurityContext:
  seccompProfile:
    type: RuntimeDefault
EOF
helm upgrade --install --wait --timeout 15m --atomic \
  --namespace kyverno --create-namespace \
  --repo https://kyverno.github.io/policy-reporter \
  policy-reporter policy-reporter --values - <<EOF
securityContext:
  seccompProfile:
    type: RuntimeDefault
kyvernoPlugin:
  enabled: true
  securityContext:
    seccompProfile:
      type: RuntimeDefault
ui:
  enabled: true
  plugins:
    kyverno: true
  securityContext:
    seccompProfile:
      type: RuntimeDefault
EOF

Once updated, the policy reporter dashboard should not report restrict-seccomp-strict related violations anymore 🎉

The last violation is about capabilities and has been fixed in Kyverno main branch but was not released yet.

Wrapping it up

This story shows how to deploy and use various Kyverno components and how to use policy-reporter to help fixing policy violations.

The violations here affect Kyverno components and the good news is that they could be fixed in Kyverno directly.

I opened pull requests to set the correct security context. Hopefully, Kyverno components will be compatible with Pod Security Standards (Restricted) out of the box once those PRs are merged.

fix: seccomp profile by eddycharly · Pull Request #3313 · kyverno/kyverno

feat: add seccomp profile support by eddycharly · Pull Request #120 · kyverno/policy-reporter

Update: The policy-reporter PR was merged and released 🎉