Kubernetes: Security assessment guidelines and necessary checklist

Introduction

The Kubernetes platform has become increasingly popular for managing containerized applications, making it a critical component of many organizations’ infrastructure.

With the increasing adoption of Kubernetes, the importance of ensuring its security cannot be overstated.

Kubernetes security measures refer to a comprehensive set of guidelines and criteria aimed at ensuring the security of a Kubernetes deployment.

It also covers various aspects of security such as network security, access control, image signing, and more.

🎯 Goals & Objectives:

The purpose of this checklist is to provide a systematic approach to securing a Kubernetes cluster and to ensure that all necessary steps are taken to protect the cluster from potential threats.

In a nutshell, it provides a starting point for organizations to evaluate the security of their Kubernetes deployment and make any necessary changes to ensure its safety.

Security Checklist

The following are some of the items that may be included in a Kubernetes security checklist:

1. Authentication

• Using an identity provider (IdP) such as OIDC is recommended for authentication to the Kubernetes API, as it provides a centralized and secure way to manage user identities.

Service account tokens, on the other hand, are less secure and should not be used for authentication. Instead, using an IdP helps ensure that the authentication process is robust and follows industry standards for security.

• Using a centralized certificate management service is a best practice for managing certificates within a Kubernetes cluster. It provides a centralized and secure way to manage and distribute certificates, ensuring that all components within the cluster use the correct and up-to-date certificates. This helps to ensure the security and trustworthiness of communication within the cluster.
• It is important to have personalized user accounts in a Kubernetes cluster. This helps to ensure accountability and traceability of actions within the cluster. Naming the service accounts in a way that reflects their purpose and access rights is also a best practice, as it makes it easier to understand the intended use of each account and to manage permissions effectively.

2. Authorization

• Implement a role-based access model that can help to simplify the management of user permissions and reduces the risk of misconfigured permissions, making the cluster more secure and easier to manage.
• Role-Based Access Control(RBAC)should be Implemented for the Kubernetes cluster. Rights need to be assigned within the namespace based on least privilege and separation of duties (RBAC-tool).
• All services should have a unique service account with configured RBAC rights.
• Enforce a formal approval process for granting access to a production environment in a Kubernetes cluster. This helps to ensure that only authorized users have access to sensitive production data and systems, reducing the risk of security breaches or unauthorized changes.
• Prohibit user-impersonation in a Kubernetes cluster. User impersonation refers to the ability to perform actions under another user’s account, which can lead to security and accountability issues
• Cluster administrators and maintainers use privileged access management systems, such as Teleport or Boundary, when interacting with the cluster API and infrastructure services. These systems provide a secure and auditable way to access the cluster, helping to ensure that sensitive data and systems are not compromised.
• Divide information systems into separate namespaces is a best practice in Kubernetes. This helps to enforce security boundaries between different systems, reducing the risk of unauthorized access or changes. It also makes it easier to manage and control access to each system.
• Audit regularly the role-based access control (RBAC) rights within a Kubernetes cluster. Tools like Kubiscan or Krane can be used to automate this process, helping to ensure that RBAC rights are up-to-date and comply with security policies.

3. Secrets

• Store secrets in a secure manner within a Kubernetes cluster. This can be done by using third-party secret management systems such as Hashicorp Vault or Conjur, or by storing secrets in etcd in encrypted form. This helps to protect sensitive information, such as passwords, API keys, and other secrets, from unauthorized access or change.
• Use secure mechanisms for adding secrets to containers in a Kubernetes cluster. This can be done by using the volumeMount mechanism or the secretKeyRef mechanism.
• use tools like the sealed-secrettool to hide secrets in source code, which can help encrypt secrets and securely store them. This helps to ensure that secrets are not exposed in the source code or version control systems, reducing the risk of unauthorized access or changes.

4. Cluster Configuration Security

• Use mutual TLS encryption between all cluster components.
• Deploy service mesh which will reduce the risk of security incidents and improve the reliability and performance of your applications. Consider using an established service mesh solution, such as Istio or Linkerd, to help ensure that your service mesh is well-supported and reliable.
• Use a policy engine to enforce security and compliance policies within a Kubernetes cluster. A policy engine allows you to define, enforce, and manage policies for your cluster. There are several policy engines to choose from, including OPA, Kyverno, jsPolicy, and kuberwarden.
• Complying with the CIS (Center for Internet Security) Benchmarkfor Kubernetes is a best practice for securing a Kubernetes cluster. The CIS Benchmark provides a set of guidelines and best practices for securing a Kubernetes cluster.
• It is recommended to use only the latest versions of cluster components (CVE list).
• For services with increased security requirements, it is recommended to use a low-level run-time with a high degree of isolation (gVisor, Kata-runtime ).
• Regular auditing of the cluster configuration is a critical aspect of maintaining the security and stability of a Kubernetes cluster. Regular auditing can help identify any potential security vulnerabilities or misconfigurations in the cluster.

Several tools can be used to audit the configuration of a Kubernetes cluster regularly such as Kube-bench, Kube-hunter, Kubestriker

5. Audit and Logging

• Enable audit logging cluster-wide.
• Log all cases of changing access rights in the cluster.
• log all operations with secrets. This can include events such as secret creation, deletion, access, and modification. You can use tools such as syslog, Fluentd, or ELK Stack to aggregate and analyze your logs.
• The audit logging system should be located outside the Kubernetes cluster.
• Building observability and visibility processes in the infrastructure and services can help understand what is happening and identify issues in real time. We can use Luntry or DataDog or WeaveScope
• Use third-party security monitoring tools on all cluster nodes (Falco, SysDig, Aqua Enterprise, NeuVector, Prisma cloud compute).

6. Secure OS configuration

• Host administrators and maintainers should interact with cluster nodes through privileged access management systems (PAM) to ensure the security and integrity of the cluster. PAM systems provide a secure and controlled way of accessing the cluster nodes and help to prevent unauthorized access or manipulation.
• Configure the operating system (OS) and software following established baselines and standards to ensure the security, stability, and compatibility of the systems. CISand NIST can help achieve this.
• It is advisable to frequently check packages and setups for security weaknesses using tools such as OpenSCAP profilesor even Lynis.
• It is suggested to keep the OS kernel version updated regularly. CVEhound can be helpful.

7. Network Security

• All namespaces should have NetworkPolicy: communication between namespaces should be controlled through NetworkPolicy and follow the principle of least privilege(Inspektor Gadget).
• Ensure that only authorized users and services can access the resources by implementing authentication and authorization for all application microservices. Check Consul it can help to fulfill this practice.
• It is recommended to keep the interfaces of cluster components and infrastructure tools inaccessible from the Internet.

8. Secure configuration of workloads

• You must not run pods under root account ie. UID 0.
• Set runAsUser parameter for all applications.
• Set allowPrivilegeEscalation - false.
• Do not run the privileged pod(privileged: true).
• It is a best practice to set readonlyRootFilesystem - true.
• It is not recommended to use hostPID and hostIPC.
• The use ofhostNetwork is a bad idea.
• You must not use unsafe system calls (sysctl) such as kernel.shm *, kernel.msg *, kernel.sem, fs.mqueue. *
• Do not use hostPath.
• Use CPU / RAM limits. The values should be the minimum for the containerized application to work.
• Capabilities should be set according to the principle of least privileges. Drop ‘ALL’, after which all the necessary capacities for the application to work are enumerated, while it is prohibited to use:
  - CAP_FSETID
  - CAP_SETUID
  - CAP_SETGID
  - CAP_SYS_CHROOT
  - CAP_SYS_PTRACE
  - CAP_CHOWN
  - CAP_NET_RAW
  - CAP_NET_ADMIN
  - CAP_NET_BIND_SERVICE
• The application should have a seccomp, apparmor, or selinux profile according to the principles of least privileges (Udica, Oci-seccomp-bpf-hook, Go2seccomp, Security Profiles Operator).
• Workload configuration should be audited regularly (Kics, Kubeaudit,Kubescape, Conftest, Kubesec, Checkov)

9. Secure image build

• Avoid using the RUN command with sudo.
• COPY is required instead of ADD instruction.
• Avoid automatic package update via apt-get upgrade, yum update, apt-get dist-upgrade.
• It is mandatory to explicitly indicate the versions of the installed packages. The SBOM building tools (Syft) can be used to determine the list of packages.
• Do not store sensitive information (passwords, tokens, certificates) in the Dockerfile.
• The composition of the packages in the container image should be minimal enough to work.
• The port range forwarded into the container should be minimal enough to work.
• It is not recommended to install wget, curl, and netcat inside the production application image and container.
• It is recommended to use dockerignore to prevent putting sensitive information inside the image.
• It is recommended to use a minimum number of layers using a multi stage build.
• It is recommended to use WORKDIR as an absolute path. It is not recommended to use cd instead of WORKDIR.
• It is recommended to beware of recursive copying using COPY . ..
• It is not recommended to use the latest tag.
• Do not run remote control tools in a container.
• Based on the results of scanning Docker images, an image signature should be generated, which will be verified before deployment (Notary, Cosign).
• Dockerfile should be checked during development by automated scanners (Kics, Conftest, Container-security-checklist ,Hadolint).
• All images should be checked in the application lifecycle by automated scanners (Trivy, Clair, Grype).
• Build secure CI and CD as same as the supply chain process (SLSA)

🌟Conclusion 🌟

Follow the best practices compiled in these lists and you’ll have taken the most important steps to successfully hardening your Kubernetes environments and protecting your critical business applications.

Thank you for Reading !! 🙌🏻😁📃, see you in the next blog.🤘

🚀 Feel free to connect with me :

LinkedIn: https://www.linkedin.com/in/rajhi-saif/

Twitter : https://twitter.com/rajhisaifeddine

The end ✌🏻

Resources:

Kubernetes Security : 6 Best Practices for 4C Security Model

Docker security

Top 20 Docker Security Best Practices: Ultimate Guide

Kubernetes Security Best Practices: 10 Steps to Securing K8s

Securing a Cluster

Kubernetes Security - OWASP Cheat Sheet Series