Policy Enforcement in Kubernetes: Restricting kubectl exec with Gatekeeper

Elevate Your Security Strategy: Advanced Techniques for Restricting Kubernetes Access

Kubernetes has emerged as the de facto standard for container orchestration, streamlining the deployment, scaling, and operations of application containers across clusters of hosts. However, with great power comes great responsibility, especially in terms of security and access control. In this blog, we explore a crucial aspect of Kubernetes security — restricting the use of kubectl exec. We will explore why this restriction is necessary and how Gatekeeper, an open-source policy engine, can help enforce these policies effectively.

Why Restrict kubectl exec?

The kubectl exec command is incredibly powerful, allowing users to spawn a process in a running container. This capability, while useful for troubleshooting, poses significant security risks. It can be a gateway for malicious activities, such as:

1. Privilege Escalation: Malicious users can exploit kubectl exec to gain elevated privileges within a container, potentially compromising the entire cluster.
2. Data Breach: Unauthorized access to containers can lead to exposure of sensitive data.
3. Breaking Compliance: Unregulated access can lead to non-compliance with regulatory standards that demand strict access controls and logging.
4. Operational Risks: Misuse of kubectl exec can unintentionally disrupt critical processes within containers, affecting application performance and stability.

Given these risks, it’s imperative to have a mechanism to control and monitor the use of kubectl exec across Kubernetes clusters.

Understanding the Limitations of RBAC in Command-Specific Restrictions

The Role of RBAC

RBAC is a fundamental tool in Kubernetes for managing who can access various resources. It’s effective for controlling actions like kubectl exec, but it doesn't govern what actions are performed after access is granted.

The Need for Command-Specific Restrictions

In one of my recent projects, I encountered a scenario where the requirement was not just to restrict who could use kubectl exec, but also to limit the specific commands that could be executed post-access. This requires a more specific control that goes beyond the capabilities of RBAC.

For example, you might want to allow developers to access containers for troubleshooting (using commands like ls, cat, etc.) but prevent them from running potentially disruptive commands. This level of control requires a policy-driven approach that can inspect and govern the actions performed within a kubectl exec session.

Gatekeeper as a Solution

Gatekeeper extends the capabilities of RBAC by enabling command-specific restrictions within a kubectl exec session, offering a more granular level of control.

Gatekeeper uses Kubernetes Custom Resources (Constraints and Templates) to enforce policies

Introducing Gatekeeper

Gatekeeper is an open-source policy engine that helps enforce policies in a Kubernetes environment. It’s built around the Open Policy Agent (OPA), a general-purpose policy engine, and extends Kubernetes to provide a native, declarative approach to policy enforcement. Gatekeeper allows you to:

• Define Custom Policies: Create policies tailored to your organization’s security needs.
• Enforce Policies at Admission Time: Ensure that resources in the Kubernetes cluster adhere to the defined policies before they are admitted.
• Audit Existing Resources: Check existing resources in the cluster against the defined policies.

Step-by-Step Guide to Restrict kubectl exec Using Gatekeeper

Step 1: Configure Gatekeeper Settings

Before installing Gatekeeper, it’s important to adjust its settings to include the CONNECT operation. By default, Gatekeeper settings include only CREATE and UPDATE operations. The CONNECT operation is essential for policies that involve actions like kubectl exec.

You can see this commit as reference for this step. You can create your own fork or clone the repo and modify the settings based in your requirements.

Step 2: Install Gatekeeper

With the configuration ready, install Gatekeeper in your Kubernetes cluster.

Note: This installation command uses a forked YAML that includes the CONNECT operation. For more detailed instructions and options, refer to the Gatekeeper documentation.

kubectl apply -f https://raw.githubusercontent.com/jcanizalez/gatekeeper/master/deploy/gatekeeper.yaml

Step 3: Define a Constraint Template

Create a Constraint Template with the specific policy to restrict kubectl exec commands. This policy blocks interactive kubectl exec and allows only certain commands like ls and cat.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8sdenyexec
spec:
  crd:
    spec:
      names:
        kind: k8sdenyexec
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sdenyexec
        import future.keywords.every
        violation[{"msg": msg}] {
          input.review.object.stdin == true
          msg :=  "Interactive exec is not permitted."
        }
        allowed_commands := {"ls","cat"}
        violation[{"msg": msg}] {
          some command in input.review.object.command
            not command in allowed_commands
          msg := sprintf("Only %v commands are allowed.", [allowed_commands])
        }

Be aware that in Linux, commands can be concatenated, which could potentially be used to bypass restrictions. The provided Rego policy is an example and should be further refined to validate command arguments. This is crucial to prevent users from injecting unauthorized commands.

Step 4: Apply the Constraint

Once the template is in place, apply a constraint that uses this template to enforce the policy.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: k8sdenyexec
metadata:
  name: k8sdenyexec
spec:
  match:
    kinds:
      - apiGroups: [ "" ]
        kinds: [ "PodExecOptions" ]

Step 5: Test the Policy

To verify the effectiveness of your policy, follow these steps:

1. Create a New Pod: Start by creating a new pod in your Kubernetes cluster. For example, you can deploy a simple Nginx pod:

kubectl run nginx - image=nginx

2. Attempt Interactive kubectl exec: Try executing an interactive command in the Nginx pod:

kubectl exec -it nginx - /bin/bash

You should encounter an error message indicating that interactive exec is not permitted:

Error from server (Forbidden): admission webhook "validation.gatekeeper.sh" denied the request: [k8sdenyexec] 
Interactive exec is not permitted.

This confirms that the policy to block interactive kubectl exec is working as intended.

3. Execute an Allowed Command in Non-Interactive Mode: Next, test a non-interactive execution of an allowed command, such as ls:

kubectl exec nginx - ls

You should see the output of the ls command, as it is included in the allowed_commands list in your policy.

4. Attempt a Restricted Command: Finally, try executing a command that is not in the allowed_commands list, such as ifconfig:

kubectl exec nginx - ifconfig

The expected response should be an error message, similar to the following:

Error from server (Forbidden): admission webhook "validation.gatekeeper.sh" denied the request: [k8sdenyexec] 
Only {"cat", "ls"} commands are allowed.

This indicates that the policy is correctly restricting commands not explicitly allowed.

Conclusion

Restricting kubectl exec in Kubernetes is crucial for maintaining a secure and compliant environment. Gatekeeper offers a robust and flexible way to enforce such policies, ensuring that only authorized operations are performed within your Kubernetes clusters. By following the steps outlined in this guide, you can start leveraging Gatekeeper to safeguard your Kubernetes environment, striking the right balance between operational flexibility and security. Remember, in the world of Kubernetes, security is not an afterthought, but a fundamental aspect of cluster management and architecture.