This summer I had the chance to work on an exciting feature at Kyverno as part of the LFX Mentorship.
This blog post is a record of my experiences and serves as a documention of my contributions.
Acknowledgment
First, let me express my gratitude to my mentor Shuting Zhao, Senior software engineer at Nirmata, for her patience, kindness and guidance throughout the mentorship program.
I also want to thank Jim Bugwadia, Co-founder and CEO at Nirmata, for his inputs and guidance, and Chip Zoller, Technical Product Manager at Nirmata, for assisting me during my demo at a Kubernetes #sig-auth meeting.
Thanks to the Linux Foundation and the Cloud Native Computing Foundation for offering this amazing mentorship program and giving us the opportunity to work on open source projects.
Introduction
About me
I’m a final year student majoring in computer science. I’m passionate about cloud computing and cloud native projects.
LFX Mentorship
The LFX mentorship is a remote learning opportunity for the open-source contributors who will be working for 12 weeks under the guidance of mentors who are maintainers and developers of the particular project and they help the mentees to contribute to the community and project.
This mentorship program is organized thrice a year i.e. Spring, Summer, and Fall months. The CNCF maintains a repository with all the information you need including the participating projects and the required skills.
How did I learn about the LFX mentorship program?
I was grateful to be a Dan Kohn scholarship recipient to attend the KubeCon + CloudNativeCon Europe 2022.
Some of my friends used Kyverno in their projects, so we stopped by the Kyverno booth where I had the chance to speak with project maintainers. I had already heard of the LFX mentorship program but one prior LFX mentee at Kyverno, Anushka Mittal, shared with me her experience and recommended me to give it a shot.
Application
I checked the available Kyverno projects and the recommended skills for the Summer 2022 cycle in the LFX Mentorship repository.
One project particularly caught my eyes: Integrate Kubernetes Pod Security with Kyverno because the project might have a significant impact on the entire cloud native community and I already had some basic knowledge in Kubernetes and Go.
I had to submit 2 documents to apply in the LFX Platform:
- Resume
- Cover Letter
In the cover letter, I made sure to highlight that I’ve used Kyverno, went through the codebase, and my comprehension of the issue that this project seeks to solve. Additionally, I quickly mentioned a few projects where I applied the recommended skills: Go application development and Kubernetes.
After two weeks, I received the following email from the Linux Foundation stating that I was accepted as a LFX mentee at Kyverno:
Project: Extend Pod Security Admission
Introduction
Kubernetes v1.25 was released on August 23rd 2022 and one major change is the removal of PodSecurityPolicy and the graduation of Pod Security Admission to stable.
The official release note states that:
PodSecurityPolicy was initially deprecated in v1.21, and with the release of v1.25, it has been removed.
The updates required to improve its usability would have introduced breaking changes, so it became necessary to remove it in favor of a more friendly replacement.
That replacement is Pod Security Admission, which graduates to Stable with this release. If you are currently relying on PodSecurityPolicy, please follow the instructions for migration to Pod Security Admission.
Problem
Once Pod Security Admission is enabled for namespaces, a configured level of Privileged, Baseline, or Restricted applies to all pods and workloads within the namespace. The level is configured as a label on the namespace. There is no option to select specific pods or control, for granular policies.
Users can choose to configure controls for select pods, but there is no validation or enforcement beyond meeting the requirements for the specified level. This means that if a namespace is baseline, any pod in that namespace can run with root user privileges.
Solution
Kyverno
Kyverno is a Kubernetes-native policy management engine with which you can define logical policies across your Kubernetes cluster around validating, mutating and generative behaviour.
PSA is an in-tree solution for pod security compliance, and its best for users to leverage the built-in solution for pod security enforcement. Kyverno can be integrated with PSA to extend its ability by providing fine grained checks and other functions.
With this solution, Kyverno can reuse the in-tree pod security implementation and not require users to manage multiple policies, one for each control defined in the pod security standards.
Extend PSA with a new validation rule
Kyverno can extend PSS profiles with a new validation rule: `validate.podSecurity`.
The rule will contain a `level` attribute that is set to a Pod Security Standard profile and a list of exclusions that the user wants to allow. When set to `restricted`, Kyverno will apply both Baseline and Restricted pod security profiles and report violations.
If the PSA is enabled with “enforce=restricted” across the cluster, there’s not much left to do. However, this is typically not the case. When PSA is in “enforce=baseline” or “enforce=privileged” mode, users can use the Kyverno rule to manage pod security in a granular fashion.
The podSecurity validate rule applies configured pod security profile level to the selected namespaces. You can define exemptions from enforcement when creating pods. Exemptions can be configured via the podSecurity.exclude attribute.
We use container images as a selector for pod security controls that are declared at the container-level
Example 1: Exclude a control
For example, if a user wants to enforce restricted PSS level to the selected namespaces “test” and “staging” but to skip checking the control “Capabilities” for pods running “ghcr.io/example/nginx:1.2.3”, the following policy does the job.
Example 2: Exclude a control’s specific restricted fields
If the user wants to specifically skip checking the custom “add” capabilities, restrictedField can be used to define the allowed list. For example, the following rule applies restricted PSS level to namespaces “test” and “staging” while excluding pods running image “ghcr.io/example/nginx:1.2.3” to be able to add kill, SETGID and SETUID capabilities.
Example 3: Fine grained control over Pod Security Admission
Let’s create 2 namespaces to demonstrate the use case of the new validation rule:
- baseline: enforce baseline Pod Security Standard level
- privileged-with-kyverno: enforce privileged Pod Security Standard level
Let’s try to run the following pod in these namespaces.
For `baseline` namespace, we get the following error message that forbids the creation of the pod because `securityContext.privileged` was set to true for the 2 containers.
Before trying to run the same pod in `privileged-with-kyverno`, let’s apply a Kyverno rule that will exclude these checks.
If we try to run the pod, it’s allowed thanks to the Kyverno rule.
Let’s test that our Kyverno policy enforces the baseline PSS level to containers that are not matching the `exclude.images`. I changed the version of the nginx docker image.
If we try to run this pod, Kyverno returns the following error and its creation is forbidden:
Organization
I had a weekly follow up meeting with my mentor to discuss the work done and plan the tasks to do for the next meeting. I highly recommend you to have a file with a progress tracker and where you keep meeting notes and share it with your mentor. It serves as a record of discussions for future reference.
The main mode of communication with my mentor and Kyverno maintainers was Slack.
The 1st month was dedicated to understanding Pod Security Admission, going through the source repository and coming up with a first prototype for the new `validate.PodSecurity` Kyverno rule.
After that period, I spent roughly 5 weeks to implement all the Pod Security Standard control and the related unit tests.
Then I created a Pull Request for the feature. I resolved code reviews from Kyverno maintainers and fixed the last issues.
Contributions
Here is the list of contributions I made throughout the program:
- Pull Request: Extend Pod Security Admission
- Demos at Kyverno Contributors meeting:
- First demo on July 20th, 2022, Second demo on August 24th, 2022
- Demo at a Kubernetes sig-auth meeting
Open issues:
- kyverno/website documentation: [Enhancement] Extend Pod Security Admission
- kyverno/kyverno CLI: [Feature] CLI: add support for validate.podSecurity rule
- kubernetes/kubernetes: Formalized CheckResult
Graduation, what’s next ?
It was an insightful and challenging experience to work on this project. Big thanks again to all Kyverno maintainers for their help and warm support.
I consider this mentorship program as the starting point of my open source journey and will definitely continue to contribute Kyverno and other projects.
Thanks for reading and connect with me on:
