2017 is becoming the year of Container Security

From https://blog.docker.com/2016/02/docker-engine-1-10-security/

Security is becoming an important — if not the most important — feature of containerization in 2017. Not just for companies like Docker, but for Linux kernel maintainers, cloud vendors, Linux OS distributions and of course, container orchestrators such as Swarm and Kubernetes.

Security is always of great importance, but that doesn’t mean it gets the marketing and media attention it deserves. Interestingly, however, I think, Docker and the rise of mass-scale containerisation outside of Silicon Valley is putting security front and centre.

Container security in 2017

Everyone has a different take on containerisation and security but I’d like to offer this viewpoint. Security is becoming the focus not because container technology itself has made apps or hosts more vulnerable, but in many cases, the opposite. And smart companies are paying attention and changing their story, their marketing and as a result, their engineering efforts.

When containerisation is implemented with good security practices, containers can offer better application security rather than a VM only solution. This is because there is now the opportunity for the container to be an additional boundary between an application exploit occurring and the attacker getting access to the host.

Security folks will point out that there are several types of attacks and that the above example is listing only one type. Fair, and true. But my main point is that if containers conceptually give us an extra line of defence, then let’s stop focussing on how containers are security risks and, instead, turn them into a new defence mechanism.

Containerisation is forcing container adopters to enforce good security practices

The explosion of the containerisation of applications combined with the push of breaking monolithic applications into micro-services (right or wrong), means that regardless of the intent, this paradigm shift is causing distributed systems to become mainstream. This is not because of a desire to implement a distributed system, but simply because that’s what you end up with, especially once you add a container orchestrator into the mix.

As a result, Developers and Operations folk who may have previously supported a single large application will now need to start becoming better informed about concepts such as the principle of least privilege, so if one node of their cluster becomes compromised, this doesn’t mean the whole system does.

Perhaps this is why Security folk are now being brought into conversations earlier (shifting to the left as they say), as the level of risk rises with the increased complexity of a system with more moving parts.

As Uncle Ben tells Peter Parker in Spider-Man “With great power, comes great responsibility”, so does implementing containers and container orchestration.

A concrete example: security by default in Docker Swarm

At the recent Container Camp Conference held in Sydney Australia this May, Diogo Mónica (Security Lead at Docker) explained the lengths that Docker Engineering is going to, to make sure that Swarm (their container orchestration tool) is secure by default, out of the box.

A Docker “swarm” is a cluster of virtual machines with one of two roles, a privileged manager and an un-privileged worker.

Swarm puts the principle of least privilege and security into action by default in many ways — three of which I think deserve the most focus.

1. Swarm encrypts node-to-node communication by default

  • Mutual TLS by default. When you start a Swarm cluster, a new self-signed CA certification is generated.
  • Nodes joining the Swarm cluster get certificates created, rotated and set up for free.
  • Certificate re-generation and rotation can happen as often as once per hour or on-demand.

In short, Swarm ensures that nodes communicate over TLS always, by default.

Of course this can be done in Kubernetes and other orchestrators but requires additional administration effort to do so.

2. Swarm secrets management: least privilege and encryption by default

  • Secrets are encrypted on the worker node (encrypted at rest) in a RAM disk (memory but on file system).
  • Worker nodes are only delivered the secrets they need from a manager.
  • Works to least privilege as workers only get what they need. No secrets needed? No secrets given.
  • When a container is removed from a node, the secrets are removed from memory as well.

When your container orchestrator provides out of the box support for encrypted secrets, and the ability to ensure only the nodes that need specific secrets will receive them, then Development and Operations have no excuses for secrets management best practices.

3. Swarm and more least privilege examples

Swarm is built with the “not if, but when” intrusion perspective. The question then becomes how to isolate a compromised node (manager or worker), limiting the damage done and ensuring that the cluster can even self-heal.

  • Swarm worker nodes are unable to get management node metadata or keys.
  • Swarm worker nodes have their secrets stored in memory so even if the compromised VM is exported, the secrets will not be part of the snapshot.
  • Swarm worker nodes do not blindly trust their manager nodes. For example, each distributed container image is signed and checked to ensure that the image they get is what they expected.

Kubernetes and other orchestration tools also offer many of these features but it’s my opinion that Docker Swarm is currently leading the field in terms of making good security practices easy by simply making them the default configuration and having them work out of the box.

4. Container OS specific images and least privilege

Docker — through its Open Source LinuxKit project — allows teams to construct their own tiny and incredibly restricted virtual machine images in minutes, bundling only what the containers will need at runtime.

5. Container image security and least privilege

Docker and the 17.05 multi-stage build feature make it easy for Developers and Operations to construct images that only have the bare essentials inside the container for runtime execution.

Container Runtime security leaping ahead

The Open Source container runtime runc project recently had code merged that allows it to run as an unprivileged user.

SUSE is also making huge strides in both enabling a non-privileged user to build and run containers, but not without some host OS limitations.

Container security is now rapidly improving across the board

Hopefully you’re now cautiously excited by the huge improvements that are arriving in container image, runtime and orchestration security.

From the Linux kernel up, a huge amount of effort is being invested into turning containers’ once-concerning security story into one of its best features, while educating non-security folk about best practices along the way.


Big thank-you to Docker Captain Scotty Coulton and Docker Melbourne Meetup organiser Ben Griffin for reviewing this article.