OWASP Kubernetes Top 10: A Comprehensive Guide

23 min readSep 25, 2023

Protect Your Kubernetes Clusters from Attack s

📌 Overview:

Kubernetes is a powerful container orchestration platform that has revolutionized the way we develop and deploy applications. However, with great power comes great responsibility, and Kubernetes security is a top concern for many organizations.

The OWASP Kubernetes Top 10 is a prioritized list of the most common security risks associated with Kubernetes. It is an essential resource for security practitioners, system administrators, and software developers alike.

In this blog post, we will take a comprehensive look at the OWASP Kubernetes Top 10. We will discuss each risk in detail, and provide recommendations for mitigating it. We will also group some of the risks together into categories, such as misconfigurations, monitoring, and vulnerabilities. Finally, we will recommend some tools and techniques for auditing your configuration and making sure that your security posture is the most appropriate.

We hope that this blog post will help you to understand and mitigate the security risks associated with Kubernetes.

In the visual above, we spotlight which component or part is impacted by each of the risks that appear in OWASP Kubernetes mapped to a generalized Kubernetes threat model to aid in understanding. This analysis also dives into each OWASP risk, providing technical details on why the threat is prominent, as well as common mitigations. It’s also helpful to group the risks into three categories and in order of likelihood.

🕸 What Does OWASP Kubernetes Refer To?:

The OWASP (Open Web Application Security Project) is a non-profit organization dedicated to enhancing software security. While initially concentrating on web application security, OWASP has expanded its focus due to evolving system designs.

As software development transitions from traditional monolithic architectures hosted on virtual machines shielded behind firewalls to contemporary microservice setups running on cloud infrastructure, it becomes imperative to adapt security measures to suit each unique application environment.

For this reason, the OWASP Foundation has introduced the OWASP Kubernetes Top 10, comprising the ten most prevalent attack vectors tailored specifically for the Kubernetes environment.

⚛️ Misconfigurations:

Insecure Workload Configurations:

The security context of a workload in Kubernetes is highly configurable which can lead to serious security misconfigurations propagating across an organization’s workloads and clusters. The Kubernetes adoption, security, and market trends report 2022 from Redhat stated that nearly 53% of respondents have experienced a misconfiguration incident in their Kubernetes environments in the last 12 months.

Cloud service providers such as AWS, GCP, and Azure implement an array of sandboxing features, virtual firewall features, and automatic updates to underlying services to ensure your business stays secure whenever and wherever possible. These measures also alleviate some of the traditional security burdens of on-premises environments. However, the cloud environments apply what is known as a shared security model, which means part of the responsibility is on the cloud service consumer to implement these security guardrails in their response environment. Responsibilities also vary based on the cloud consumption model and type of offering.

To ensure the security of cloud-native workloads, administrators must prioritize safe image usage, maintain updated operating systems, and conduct continuous audits of infrastructure configurations. Misconfigurations are a prevalent entry point for adversaries seeking unauthorized access.

✅ Operating System:

Containerized workloads offer the advantage of preloaded dependencies aligned with the chosen application’s base image for a specific operating system (OS). However, these images often include unnecessary system libraries and third-party components, potentially leading to bloated containers, particularly within microservices architecture.

To mitigate this, it’s advisable to opt for minimal, streamlined images like Alpine Linux, known for their smaller file size. These lightweight images not only enhance performance but also reduce the potential points of compromise. For cases requiring additional packages or libraries, it’s recommended to start with the base Alpine image and gradually incorporate them while maintaining the desired behavior and performance.

✅ Audit workloads:

The CIS Benchmark for Kubernetes can be used as a starting point for discovering misconfigurations. The open source project kube-bench, for instance, can check your cluster against the (CIS) Kubernetes Benchmark using YAML files to set up the tests.

✅ Preventing Workload Misconfigurations with OPA:

Workload misconfigurations can be a significant security risk in cloud-native environments. Fortunately, tools like the Open Policy Agent (OPA) offer a powerful solution. Acting as a policy engine, OPA provides the means to identify and rectify common misconfigurations. Its admission controller offers a declarative language to create and enforce policies throughout your stack.

Suppose you need to develop an admission controller for an Alpine image, as mentioned earlier. However, a Kubernetes user wants to configure the securityContext with privileged=true. OPA can help address this scenario effectively. By leveraging OPA, you can establish policies that automatically flag and prevent such misconfigurations, enhancing the security posture of your Kubernetes environment.

Misconfigured Cluster Components:

Surprisingly common in core Kubernetes components, misconfigurations pose a significant threat to the security of your environment. To counteract this, implementing a system of continuous and automatic auditing for both Infrastructure as Code (IaC) and Kubernetes YAML manifests is crucial. This approach eliminates the need for manual checks and significantly reduces the likelihood of configuration errors. By automating this process, you not only enhance the security posture of your Kubernetes deployment but also free up valuable time for your team to focus on other critical tasks.

Misconfigurations in core Kubernetes components can lead to complete cluster compromise or worse. In this section we will explore some of the components that exist on the Kubernetes control plane and nodes which can easily be misconfigured.

One of the riskiest misconfigurations is the Anonymous Authentication setting in Kubelet, which allows non-authenticated requests to the Kubelet. It’s strongly recommended to check your Kubelet configuration and ensure the flag described below is set to false.

When auditing workloads, it’s important to keep in mind that there are different ways in which to deploy an application. With the configuration file of the various cluster components, you can authorize specific read/write permissions on those components. In the case of Kubelet, by default, all requests to the kubelet’s HTTPS endpoint that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of system:anonymous and a group of system:unauthenticated.

To disable this anonymous access for these unauthenticated requests, simply start kubelet with the feature flag –anonymous-auth=false. When auditing cluster components like kubelet, we can see that kubelet authorizes API requests using the same request attributes approach as the API Server. As a result, we can define the permissions such as:

POST
GET
PUT
PATCH
DELETE

However, there are many other cluster components to focus on, not just kubelet. For instance, kubectl plugins run with the same privileges as the kubectl command itself, so if a plugin is compromised, it could potentially be used to escalate privileges and gain access to sensitive resources in your cluster.

Based on the CIS Benchmark report for Kubernetes, we would recommend enabling the following settings for all cluster components.

✅ Securing etcd: Safeguarding Kubernetes Cluster Data:

At the heart of Kubernetes lies etcd, a highly-available key/value store responsible for centralizing cluster data. This database not only houses crucial configuration information but also safeguards sensitive data like Kubernetes Secrets. Ensuring the security of etcd is paramount to the overall integrity of your Kubernetes environment. We strongly advocate for regular backups of etcd data to mitigate the risk of potential data loss.

Fortunately, etcd offers a native snapshot feature designed for this very purpose. This feature allows you to capture a snapshot of the active cluster member using the etcdctl snapshot save command, all without incurring any performance overhead. To illustrate, here's an example of how you can take a snapshot of the keyspace served by $ENDPOINT and save it as snapshotdb:

ETCDCTL_API=3 etcdctl --endpoints $ENDPOINT snapshot save snapshotdb

✅ kube-apiserver:

The Kubernetes API server validates and configures data for the API objects, which include pods, services, ReplicationControllers, and others. The API Server services REST operations and provides the frontend to the cluster’s shared state through which all other components interact. It’s critical to cluster operation and has high value, as an attack target can’t be understated. From a security standpoint, all connections to the API server, communication made inside the Control Plane, and communication between the Control Plane and kubelet components should only be provisioned to be reachable using TLS connections.

Out of the box, the kube-apiserver lacks default TLS configuration. If flagged as a security concern in your Kube-bench results, it’s imperative to promptly address this vulnerability. To do so, enable TLS using the feature flags --tls-cert-file=[file] and --tls-private-key-file=[file] in the kube-apiserver.

Given the dynamic nature of Kubernetes clusters, it’s wise to leverage the TLS bootstrapping feature. This enables automatic certificate signing and TLS configuration within the cluster, streamlining the process and eliminating the need for manual intervention.

Regular certificate rotation is a critical best practice, particularly for long-running Kubernetes clusters. Thankfully, for Kubernetes versions 1.8 and higher, automation is available to facilitate this process.

Moreover, API server requests should undergo authentication, a topic we’ll delve into in the following section on Broken Authentication Mechanisms. Ensuring robust authentication mechanisms adds an extra layer of security to your Kubernetes environment.

✅ Enhancing DNS Security with CoreDNS in Kubernetes:

CoreDNS, an integral DNS server technology, takes the helm as the preferred DNS solution for Kubernetes clusters, surpassing kube-dns from version v1.11 onwards. In the realm of Kubernetes, reliable name resolution within a cluster is paramount for managing dynamic and short-lived workloads and services.

Notably, CoreDNS addresses critical security vulnerabilities that were present in kube-dns, particularly in dnsmasq, the DNS resolver component responsible for caching responses from SkyDNS, which ultimately handles DNS resolution services.

Beyond rectifying security concerns, CoreDNS also tackles performance bottlenecks associated with SkyDNS. Unlike kube-dns, which required an additional sidecar proxy for health monitoring and metric reporting, CoreDNS consolidates all these functions within a single container.

However, it’s crucial to recognize that no system is entirely impervious to compromise. Even with CoreDNS, vigilance is key. Regular compliance checks using tools like kube-bench remain instrumental in ensuring the continued security of your Kubernetes environment. By adopting CoreDNS, organizations bolster their DNS infrastructure, fortifying the foundation of their containerized applications.

`Overly-permissive RBAC Configurations:`

Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within your organization. An RBAC misconfiguration could allow an attacker to elevate privileges and gain full control of the entire cluster.

Creating RBAC rules is rather straightforward. For instance, to create a permissive policy to allow read-only CRUD actions (i.e., get, watch, list) for pods in the Kubernetes cluster’s default network namespace, but to prevent Create, Updated, or Delete actions against those pods, the policy would look something like this:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
Metadata:
  namespace: default
  name: pod-reader
Rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

Issues arise when managing these RBAC rules in the long run. Admins will likely need to manage ClusterRole resources to avoid building individual roles on each network namespace, as seen above. ClusterRoles allow us to build cluster-scoped rules for grant access to those workloads.

RoleBindings can then be used to bind the above mentioned roles to users.

Similar to other Identity & Access Management (IAM) practices, we need to ensure each user has the correct access to resources within Kubernetes without granting excessive permissions to individual resources. The below manifest should show how we recommend binding a role to a Service Account or user in Kubernetes.

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
Metadata:
  name: read-pods
  namespace: default
Subjects:
- kind: User
  name: nigeldouglas
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Scanning for RBAC misconfigurations not only strengthens cluster security but also simplifies permission management. Managing individual RBAC policies in production can be complex, leading to over-allocated permissions. Tools exist to handle RBAC management, audits, and compliance checks, ensuring a more efficient and secure p

✅ Audit RBAC:

RBAC Audit:

is a tool created by the team at CyberArk. This tool is designed to scan the Kubernetes cluster for risky roles within RBAC and requires python3.

This python tool can be run via a single command:

ExtensiveRoleCheck.py --clusterRole clusterroles.json \
--role Roles.json --rolebindings rolebindings.json \
--cluseterolebindings clusterrolebindings.json

The output should look somehow similar to:

Kubiscan:

Kubiscan is another tool built by the team at CyberArk. Unlike RBAC Audit, this tool is designed for scanning Kubernetes clusters for risky permissions in Kubernetes’s RBAC authorization model — not the RBAC roles. Again, Python v.3.6 or higher is required for this tool to work.

Krane:

Krane is a static analysis tool for Kubernetes RBAC. Similar to Kubiscan, it identifies potential security risks in K8s RBAC design and makes suggestions on how to mitigate them.

The major difference between these tools is the way Krane provides a dashboard of the cluster’s current RBAC security posture and lets you navigate through its definition.

Network Segmentation Controls:

By default, Kubernetes employs a “flat network” design, enabling workloads to communicate seamlessly within the cluster without explicit configuration. While convenient, this also means there are no inherent restrictions. In the event of a successful attack on a running workload, the attacker gains unrestricted access for potential data exfiltration across all other pods in the cluster.

For cluster operators prioritizing a zero trust architecture, it’s imperative to turn to Kubernetes Network Policy. This powerful tool ensures that services are appropriately restricted, establishing a crucial layer of defense against unauthorized lateral movement within the cluster. By implementing network segmentation controls, organizations can significantly enhance the security posture of their Kubernetes environment.

Kubernetes offers solutions to address the right configuration of network segmentation controls. Here, we show you two of them.

Service Mesh with Istio:

Istio provides a service mesh solution. This allows security and network teams to manage traffic flow across microservices, enforce policies, and aggregate telemetry data in order to enforce microsegmentation on the network traffic going in and out of our microservices.

At the time of writing, the service relies on implementing a set of sidecar proxies to each microservice in your cluster. However, the Istio project is looking to move to a sidecar-less approach sometime in the year.

The sidecar technology is called ‘Envoy.’ We rely on Envoy to handle ingress/egress traffic between services in the cluster and from a service to external services in the service mesh architecture. The clear advantage of using proxies is that they provide a secure microservice mesh, offering functions like traffic mirroring, discovery, rich layer-7 traffic routing, circuit breakers, policy enforcement, telemetry recording/reporting functions, and — most importantly — automatic mTLS for all communication with automatic certificate rotation!

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
Metadata:
  name: httpbin
  namespace: default
Spec:
  action: DENY
  Rules:
  - from:
    - source:
        namespaces: ["prod"]
    To:
    - operation:
        methods: ["POST"]Code language: YAML (yaml)

The above Istio AuthorizationPolicy sets action to “DENY” on all requests from the “prod” production namespace to the “POST” method on all workloads in the “default” namespace.

This policy is incredibly useful. Unlike Calico network policies that can only drop the traffic based on IP address and port at the L3/L4 (network layer), the authorization policy is denying the traffic based on HTTP/S verbs such as POST/GET at L7 (application layer). This is important when implementing a Web Application Firewall (WAF).

CNI:

It’s worth noting that although there are huge advantages to a service mesh, such as encryption of traffic between workloads via Mutual TLS (mTLS) as well as HTTP/s traffic controls, there are also some complexities to managing a service mesh. The use of sidecars beside each workload adds additional overhead in your cluster, as well as unwanted issues troubleshooting those sidecars when they experience issues in production.

Many organizations opt to only implement the Container Network Interface (CNI) by default. The CNI, as the name suggests, is the networking interface for the cluster. CNI’s like Project Calico and Cilium come with their own policy enforcement. Whereas Istio enforces traffic controls on L7 traffic, the CNI tends to be focused more on network-layer traffic (L3/L4).

The following CiliumNetworkPolicy, as an example, limits all endpoints with the label app=frontend to only be able to emit packets using TCP on port 80, to any layer 3 destination:

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
Metadata:
  name: "l4-rule"
Spec:
  endpointSelector:
    matchLabels:
      app: frontend
  Egress:
    - toPorts:
      - ports:
        - port: "80"
          protocol: TCPCode language: YAML (yaml)

We mentioned using the Istio AuthorizationPolicy to provide WAF-like capabilities at the L7/application-layer. However, a Distributed Denial-of-Service (DDoS) attack can still happen at the network-layer if the adversary floods the pods/endpoint with excessive TCP/UDP traffic. Similarly, it can be used to prevent compromised workloads from speaking to known/malicious C2 servers based on fixed IP’s and ports.

Lack of visibility:

Inadequate Logging and Monitoring:

A Kubernetes environment has the ability to generate logs at a variety of levels from many different components. When logs are not captured, stored, or actively monitored attackers have the ability to exploit vulnerabilities while going largely undetected. The lack of logging and monitoring also presents challenges during incident investigation and response efforts.

However, there are other log sources to focus on — not limited to Kubernetes Audit Logs. They can include host-specific Operating System logs, Network Activity logs (such as DNS, which you can monitor the Kubernetes add-ons CoreDNS), and Cloud Providers that also work as the foundation for the Kubernetes Cloud.

Without a centralized tool for storing all of these sporadic log sources, we would have a hard time using them in the case of a breach. That’s where tools like Prometheus, Grafana, and Falco are useful.

Prometheus:

Prometheus is an open-source, community-driven project for monitoring modern cloud-native applications and Kubernetes. It is a graduated member of the CNCF and has an active developer and user community.

Grafana:

Like Prometheus, Grafana is an open-source tool with a large community backing. Grafana allows you to query, visualize, alert on, and understand your metrics no matter where they are stored. Users can create, explore, and share dashboards with their teams.

Falco (runtime detection):

Falco, the cloud-native runtime security project, is the de facto standard for Kubernetes threat detection. Falco detects threats at runtime by observing the behavior of your applications and containers. Falco extends threat detection across cloud environments with Falco Plugins.

Falco is the first runtime security project to join CNCF as an incubation-level project. Falco acts as a security camera, detecting unexpected behavior, intrusions, and data theft in real-time in all Kubernetes environments. Falco v.0.13 added Kubernetes Audit Events to the list of supported event sources. This is in addition to the existing support for system call events. An improved implementation of audit events was introduced in Kubernetes v1.11 and it provides a log of requests and responses to kube-apiserver.

🌉 Policy Enforcement:

Maintaining consistent security policies across multi-cluster and multi-cloud environments can be a formidable challenge. The default scenario often requires security teams to navigate and manage risks in each of these diverse environments separately. This fragmentation of efforts can lead to inefficiencies and potential gaps in security coverage. To address this, organizations should seek centralized solutions that provide a unified approach to policy enforcement, ensuring comprehensive security measures are upheld across all environments.

There’s no default way to detect, remediate, and prevent misconfigurations from a centralized location, meaning clusters could potentially be left open to compromise.

Admission controller:

An admission controller intercepts requests to the Kubernetes API Server prior to persistence. The request must first be authenticated and authorized, and then a decision is made on whether to allow the request to be performed. For example, you can create the following Admission controller configuration:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
  - name: ImagePolicyWebhook
    configuration:
      imagePolicy:
        kubeConfigFile: <path-to-kubeconfig-file>
        allowTTL: 50
        denyTTL: 50
        retryBackoff: 500
        defaultAllow: trueCode language: YAML (yaml)

The ImagePolicyWebhook configuration is referencing a kubeconfig formatted file which sets up the connection to the backend. The point of this admission controller is to ensure the backend communicates over TLS.

The allowTTL: 50 sets the amount of time in seconds to cache the approval and, similarly, the denyTTL: 50 sets the amount of time in seconds to cache the denial. Admission controllers can be used to limit requests to create, delete, modify objects, or connect to proxies.

Admission control is an important element of container security strategy to enforce policies that need Kubernetes context and create a last line of defense for your cluster. We touch on image scanning later in this research, but know that image scanning can also be enforced via a Kubernetes admission controller.

Runtime detection:

We need to standardize the deployment of security policy configurations to all clusters, if they mirror the same configuration. In the case of radically different cluster configurations, they might require uniquely designed security policies. In either instance, how do we know which security policies are deployed in each cluster environment? That’s where Falco comes into play.

Secrets Management Failures:

In Kubernetes, a Secret is a small object that contains sensitive data, like a password or token. It is necessary to assess how sensitive data such as credentials and keys are stored and accessed. Secrets are useful features in the Kubernetes ecosystem but need to be handled with extreme caution.

Regardless of this design, K8s Secrets can still be compromised. The native K8s secrets mechanism is essentially an abstraction — the data still gets stored in the aforementioned etcd database, and it’s turtles all the way down. As such, it’s important for businesses to assess how credentials and keys are stored and accessed within K8s secrets as part of a broader secrets management strategy. K8s provides other security controls, which include data-at-rest encryption, access control, and logging.

Encrypt secrets at rest:

One major weakness with the etcd database used by Kubernetes is that it contains all data accessible via the Kubernetes API, and therefore can allow an attacker extended visibility into secrets. That’s why it’s incredibly important to encrypt secrets at rest.

As of v.1.7, Kubernetes supports encryption at rest. This option will encrypt Secret resources in etcd, preventing parties that gain access to your etcd backups from viewing the content of those secrets. While this feature is currently in beta and not enabled by default, it offers an additional level of defense when backups are not encrypted or an attacker gains read access to etcd.

Here’s an example of creating the EncryptionConfiguration custom resource:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          Keys:
            - name: key1
              secret: <BASE 64 ENCODED SECRET>
      - identity: {}

Address Security Misconfigurations:

In order to keep secrets safe and protected, it is important to start with rock-solid configuration across all of your clusters. Vulnerabilities, image security, and policy enforcement need to be in place to ultimately protect the applications from compromise.

RBAC configuration should be locked down as well. Keep all Service Account and end-user access to the least privilege — especially when it comes to accessing secrets. Always audit the RBAC configuration of third-party plugins and software installed in the cluster to ensure access to Kubernetes secrets is not granted unnecessarily.

Ensure Logging and Auditing is in Place:

Kubernetes clusters generate useful metrics around activities that can help detect malicious or anomalous behavior including access to secrets. Make sure to enable and configure Kubernetes Audit records and centralize their storage.

🚦 Vulnerability Management:

Supply Chain Vulnerabilities:

Containers take on many forms at different phases of the development lifecycle supply chain; each of them presenting unique security challenges. A single container alone can rely on hundreds of third-party components and dependencies making trust of origin at each phase extremely difficult. These challenges include but are not limited to image integrity, image composition, and known software vulnerabilities.

Supply chain attacks are on the rise, as seen with the SolarWinds breach. The SolarWinds software solution ‘Orion’ was compromised by the Russian threat group APT29 (commonly known as Cozy Bear). This was a long-running zero-day attack, which means the SolarWinds customers who had Orion running in their environments were not aware of the compromise. APT29 adversaries would potentially have access to non-air gapped Orion instances via this SolarWinds exploit.

SolarWinds is just one example of a compromised solution within the enterprise security stack. In the case of Kubernetes, a single containerized workload alone can rely on hundreds of third-party components and dependencies, making trust of origin at each phase extremely difficult. These challenges include, but are not limited to, image integrity, image composition, and known software vulnerabilities.

Let’s dig deeper into each of these.

Images:

A container image represents binary data that encapsulates an application and all of its software dependencies. Container images are executable software bundles that can run standalone (once instantiated into a running container) and that make very well defined assumptions about their runtime environment.

The Sysdig Threat Research Team performed an analysis of over 250,000 Linux images in order to understand what kind of malicious payloads are hiding in the containers images on Docker Hub.

The Sysdig TRT collected malicious images based on several categories, as shown above. The analysis focused on two main categories: malicious IP addresses or domains, and secrets. Both represent threats for people downloading and deploying images that are available in public registries, such as Docker Hub, exposing their environments to high risks.

Additional guidance on image scanning can be found in the research of 12 image scanning best practices. This advice is useful whether you’re just starting to run containers and Kubernetes in production, or you want to embed more security into your current DevOps workflows.

Dependencies:

When you have a large number of resources in your cluster, you can easily lose track of all relationships between them. Even “small” clusters can have way more services than anticipated by virtue of containerization and orchestration. Keeping track of all services, resources, and dependencies is even more challenging when you’re managing distributed teams over multi-cluster or multi-cloud environments.

Kubernetes doesn’t provide a mechanism by default to visualize the dependencies between your Deployments, Services, Persistent Volume Claims (PVC’s), etc. KubeView is a great open source tool to view and audit intra-cluster dependencies. It maps out the API objects and how they are interconnected. Data is fetched in real-time from the Kubernetes API. The status of some objects (Pods, ReplicaSets, Deployments) is color-coded red/green to represent their status and health.

Registry:

The registry is a stateless, scalable server-side application that stores and lets you distribute container images.

Kubernetes resources, which implement images (such as pods, deployments, etc.), will use imagePull secrets to hold the credentials necessary to authenticate to the various image registries. Like many of the problems we have discussed in this section, there’s no inherent way to scan images for vulnerabilities in standard Kubernetes deployments.

But even on a private, dedicated image registry, you should scan images for vulnerabilities. But Kubernetes doesn’t provide a default, integrated way to do this out of the box. You should scan your images in the CI/CD pipelines used to build them as part of a shift-left security approach. See the research Shift-Left: Developer-Driven Security for more details.

Sysdig has authored detailed, technical guidance with examples on how to do it for common CI/CD services, providing another layer of security to prevent vulnerabilities in your pipelines:

Another layer of security we can add is a process of signing and verifying the images we send to our registries or repositories. This reduces supply chain attacks by ensuring authenticity and integrity. It protects our Kubernetes development and deployments, and provides better control of the inventory of containers we are running at any given time.

Broken Authentication:

Authentication in Kubernetes takes on my many forms and is extremely flexible. This emphasis on being highly configurable makes Kubernetes work in a number of different environments but also presents challenges when it comes to cluster and cloud security posture.

If it’s a person who wants to authenticate against our cluster, a main area of concern will be the credentials management. The most likely case is that they will be exposed by an accidental error, leaking in one of the configuration files such as .kubeconfig.

Inside your Kubernetes cluster, the authentication between services and machines is based on Service Accounts. It’s important to avoid using certificates for end-user authentication or Service Account tokens from outside of the cluster, because we would increase the risk. Therefore, it is recommended to continuously scan for secrets or certificates that may be exposed by mistake.

OWASP recommends that no matter what authentication mechanism is chosen, we should force humans to provide a second method of authentication. If you use a cloud IAM capability and 2FA is not enabled, for instance, we should be able to detect it at runtime in your cloud or Kubernetes environment to speed up detection and response. For this purpose, we can use Falco, an open source threat detection engine that triggers alerts at run-time according to a set of YAML formatted rules.

- rule: Console Login Without Multi Factor Authentication
  desc: Detects a console login without using MFA.
  condition: >-
    aws.eventName="ConsoleLogin" and not aws.errorCode exists and
    jevt.value[/userIdentity/type]!="AssumedRole" and
    jevt.value[/responseElements/ConsoleLogin]="Success" and
    jevt.value[/additionalEventData/MFAUsed]="No"
  output: >-
    Detected a console login without MFA (requesting user=%aws.user, requesting
    IP=%aws.sourceIP, AWS region=%aws.region)
  priority: critical
  source: aws_cloudtrail
  append: false
  exceptions: []Code language: YAML (yaml)

Falco helps us identify where insecure logins exist. In this case, it’s a login to the AWS console without MFA. However, if an adversary were able to access the cloud console without additional authorization required, they would likely be able to then access Amazon’s Elastic Kubernetes Service (EKS) via the CloudShell.

That’s why it’s important to have MFA for cluster access, as well as the managed services powering the cluster — GKE, EKS, AKS, IKS, etc.

But it is not only important to protect access to Kubernetes. If we use other tools on top of Kubernetes to, for example, monitor events, we must protect those as well. As we explained at KubeCon 2022, an attacker could exploit an exposed Prometheus instance and compromise your Kubernetes cluster.

Outdated and Vulnerable Kubernetes Components:

Vulnerabilities exist in Kubernetes and it is up to administrators to follow CVE databases, disclosures, and updates closely and have a plan for patch management.

Kubernetes admins must follow the latest up-to-date CVE databases, monitor vulnerability disclosures, and apply relevant patches where applicable. If not, Kubernetes clusters may be exposed to these known vulnerabilities that make it easier for an attacker to perform techniques to take full control of your infrastructure and potentially pivot to your cloud tenant where you’ve deployed clusters.

The large number of open-source components in Kubernetes, as well as the project release cadence, makes CVE management particularly difficult. In version 1.25 of Kubernetes, a new security feed was released to Alpha that groups and updates the list of CVEs that affect Kubernetes components.

Here is a list of the most famous ones:

CVE-2021–25735 — Kubernetes validating admission webhook bypass
CVE-2020–8554 — Unpatched Man-In-The-Middle (MITM) Attack in Kubernetes
CVE-2019–11246 — High-severity vulnerability affecting kubectl tool. If exploited, it could lead to a directory traversal.
CVE-2018–18264 — Privilege escalation through Kubernetes dashboard

To detect these vulnerable components, you should use tools that check or scan your Kubernetes cluster, such as kubescape or kubeclarity — or look to a commercial platform offering such as Sysdig Secure.

Today, the vulnerabilities released directly target the Linux Kernel, affecting the containers running on our cluster rather than the Kubernetes components themselves. Even so, we must keep an eye on each new vulnerability discovered and have a plan to mitigate the risk as soon as possible.

Conclusion:

The OWASP Kubernetes Top 10 is a valuable resource for assessing security risks within Kubernetes environments. It ranks common risks based on data from diverse organizations.

While various open-source projects can address these risks, implementing and managing them demands substantial resources and expertise. There’s no one-size-fits-all solution, but Sysdig Secure offers a holistic approach. It identifies vulnerabilities in images, container registries, and Kubernetes dependencies. By integrating with Kubernetes admission controllers, it allows or blocks vulnerable workloads. The platform also automates network threat resolution via automatic network policy generation and provides comprehensive cluster activity visibility through managed Prometheus instances.