Top security tips that Kubernetes adopters often forget about

by Nikita Mazur

Kubernetes is a very powerful platform that makes application deployment much easier compared to a traditional on-premises server or a VM. That’s probably one of the key success factors that made Kubernetes adoption boom over the last couple of years. The counterpart is that this way of doing things is relatively new, which unfortunately sometimes leads to poor implementation of good practices in terms of security. Let’s review some of the security rules and best practices to make your Kubernetes cluster stronger.

Reduce network exposure

This is probably one of the most basic principles in IT security, but unfortunately one of the least followed when it comes to using Kubernetes clusters in a public cloud.

The objective is to have a Kubernetes cluster installed in a private network, or in a public network with strong firewall rules. The reason behind it is that Kubernetes exposes many services like the API and cluster data storage, which contain sensitive information.

In addition, and in order to keep Kubernetes cluster in a private zone, it is not advisable to access the nodes directly through SSH. A good practice is to use an SSH bastion, which is basically an SSH gateway with public access and duly protected. Only the SSH traffic from this bastion will be allowed to pass to your Kubernetes cluster.

Resources isolation

All Kubernetes clusters are shared by different applications, and sometimes even by different customers and users. From a security point of view, this can be problematic for two reasons:

• The CPU/RAM resources may be monopolized by one or more containers, creating, in the best case scenario, performance issues for other applications hosted in the same cluster and, in the worst one, the complete freeze of the system.
• By default every container within a Kubernetes cluster may be reached by another container in the same cluster.

These two risks may be mitigated if you follow Kubernetes’ Resource Requests and Limits and Network Policies.

Have effective access management

For access management to be effective you must ensure that authentication is correctly implemented and that the cluster enables users to perform only the actions that they are allowed to do.

For the authentication part, Kubernetes has several methods that can be used. However, it is important to disable basic authentication with username and password access and the anonymous mode access ( — anonymous-auth=false). These two methods are not very secure.

As for the authorization part, it is advised to enable Role Based Access Control feature when starting the api server ( — authorization-mode=RBAC). It is available as a stable feature from Kubernetes 1.8.

Also, don’t forget to rotate your credentials regularly. It is a basic recommendation in any Information System.

Limit privileges in containers

One of the strongest beliefs about containers is that they are completely isolated, which is wrong. It should be reminded that a container is not a virtual machine, and that the Kernel processes are shared by all containers. As a result, containers are naturally less isolated than VMs. Even if the risk of privilege escalation from a container to a host is low because of Kubernetes and Docker security implementations, it is always advisable to limit the privileges of containers to the minimum.

Two measures can be adopted in this direction:

• Run containers as non-root: this is probably one of the least popular and less followed measures among the community. The majority of containerized apps that you can find on Docker Hub run as root simply because they are easy for the developers. However, this is clearly a potential security breach, since a process running as root in a container is actually running as root on the host node as well.
• Run containers as non-privileged. When you run a container in the privileged mode, it has access to the host resources, which is a great risk. So don’t do this unless it is really necessary.

You can implement these two recommendations through a Pod Security Context and/or a Pod Security Policy (currently in beta for Kubernetes version 1.10).

Use images from known sources

One of the most appealing features of containerized architecture is that it allows leveraging the work the community has already done, which clearly adds to the efficiency and speed of development. However, this benefit comes at a price, which is the risk to use a compromised container image. And it goes without saying that a compromised container is a compromised cluster.

That’s why it is important to use images from reliable sources. If you find an image of an app that does exactly what you need, but you don’t know the source very well, then don’t take the risk. Just copy the Dockerfile and make your own image.

Monitor your logs & perform audits regularly

This is a fundamental topic, although probably the one that implies the most effort and hassle in the long run since it is a continuous process. It consists of:

• Continuously monitoring both your application logs and your cluster logs searching for anomalies
• Performing security audits to find vulnerabilities.

Let’s check some tools you can use for these taks:

Monitoring and logging:

• ELK stack (ElasticSearch, Logstash and Kibana), which is my preferred tool. The possibilities are nearly infinite with this stack and it is open source. The counterpart is that it is a little bit difficult to setup and use at the beginning.
• Datadog: easy to install and nice UI. Free plan for customers with less than 5 nodes.
• Amazon Cloudwatch: logging solution from AWS. It would be a natural choice if you host yout Kubernetes cluster in Amazon cloud.

Auditing:

• Clair: a vulnerability scanner from CoreOS designed specifically for containers.
• Kube-bench: a tool from Aquasec that checks whether Kubernetes is deployed securely by running the checks documented in the CIS Benchmark for Kubernetes.
• OpenSCAP: a suite of automated audit tools to examine the configuration and known vulnerabilities in your software, following the NIST-certified Security Content Automation Protocol (SCAP). It has a container specific module.

Conclusion

In this article I covered the most basic security measures that should for sure help you harden your cluster. However, Kubernetes security is a very broad topic, and it would certainly need an entire course to cover all the aspects. As far as I’m concerned, I invest a lot of time in these topics with my team, because we are convinced that it’s one of the cornerstones of Kubernetes adoption.

I hope you find this useful! Feel free to ask me anything, I’ll be glad to help. Don’t forget to follow us on Twitter and join our Telegram chat to stay tuned!

You might also want to check our Containerum project on GitHub. We need you feedback to make it stronger — you can submit an issue, or just support the project by giving it a ⭐. Your support really matters to us!

Containerum is your source of knowledge on Kubernetes and Docker.