Docker Security Testing

Docker is a great and powerful technology, but, please, do not try to run it in production with the default settings. You have to make an additional work to harden your Docker based environment and make it more secure and more robust.

TL; DR

The CIS Docker 1.11.0 Benchmark paper contains a very detailed and updated information about different checks you need to do at OS level, Docker daemon level and container level. The Docker Security team also provides a Docker Bench for Security tool, that automates these configuration checks. Make sure you are using it. Continuously.

But, if you need a machine readable test result, in a standard format, with more control (which tests to run and when) and more details (report test results for individual container), you can give a try to a Docker Bench Test tool.

The Story

Lately I was looking for a tool, that would help me to verify my Docker based environment setup automatically and continuously, mainly focusing on security.

Docker, out of the box, does not require any configuration to start working with it — one command line to install and you are ready to go. This makes Docker a very appealing for developers. But it’s not recommended to run “naked” Docker in production, relying on the default configuration and with containers “baked” from different tutorial posts or taken blindly from the DockerHub registry. Docker is a great technology, but, out of the box, it is not a production ready. On the other hand, Docker provides a lot of settings, that allow you to make it much more secure and reliable.

The Docker Security Team, together with other companies and the Center for Internet Security, did a great job and released a must to read paper, called CIS Docker 1.11.0 Benchmark (it’s already the second edition of the paper). It’s a very solid and deep paper about hardening Docker, at different levels, to create secure and reliable production environment.

The Docker Security Team also did one additional thing, to help practitioners: they’ve released a Docker Bench for Security tool. The tool, that automatically validates most of the checks described in the above paper. Kudos to the team!

Docker Bench for Security is a set of Bash shell scripts, that you should run as a root user on any Docker host and the tool will produce a really nice looking and pretty formatted output with all checks. It’s very cool! You should use it, definitely. It’s almost the tool I was looking for. But…

I’m a lazy guy and do not want to go to every host to run this tool and manually inspect results produced by Docker Bench for Security scripts, plus manually validate all “Info” level messages.

Here is the list of my requirements for desired tool:

1. Automatic execution of checks described in the CIS Docker 1.11.0 Benchmark paper.
2. Machine readable test result output.
3. No “Info” statuses, statuses that require additional manual inspection. I prefer configuration over manual work.
4. Benefit: test result output should be in standard test result formats: jUnit XML or TAP, for example.
5. Execute any subset of tests or even individual test and even for a specific container.
6. Execute test per container for container level checks (most checks in section 4 and 5 in the paper).

Googling for such tool, did not return any good result. So, after looking at Docker Bench for Security scripts, I’ve decided to convert them into valid automation tests. To avoid introducing additional dependency (packages, runtime, languages, etc.), I’ve decided to look for shell based testing framework.

The Bats, Bash testing framework, is looking very promising. It allows to write test cases in standard Bash language, can produce a machine readable report in a standard TAP format, can execute individual test cases or subset of available test cases and it also provides a nice test syntax (helper functions for output validation and assertions) for writing short and readable test cases.

Docker Bench Test

And this is how Docker Bench Test project was created. I’ve converted all shell scripts from Docker Bench for Security into valid Bats test cases. Surprisingly, some tests become much shorter, than original scripts. I think, this is because of helper libraries and short and nice Bats syntax.

I also did not like the fact, that Docker Bench for Security runs container level checks as a single check. I will try to explain this point below:

Suppose, you have 20 containers running on some Docker host. And only 2 of these containers are misconfigured (running containerized process as a root user, for example). Docker Bench for Security will fail “4.1 — Create a user for the container” check, printing out the names of two containers. With Docker Bench Test, you will have 20 different test cases: 18 passed and only 2 failed.

Docker Bench Test also allows you to run any selected subset of all available tests. Why is this needed?

Different tests require different permissions. For example, if you want to check host OS configuration, you need to run your tests as root user. For testing containers, this is not required.

Also, when you operate a Dockerrized environment (some Docker cluster), the frequency of change events is different at every level. When host OS is updated of some configuration is changed, you may want to run all host level and Docker daemon level tests. Container level tests make sense to run only when a new Docker container is created.

How to run Docker Bench Test

There are two options to run Docker Bench Test test cases.

The first, and simplest option is to use Docker Bench Test as a Docker container.

$ docker run -it --net host --pid host --cap-add audit_control \
    -v /var/lib:/var/lib \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /usr/lib/systemd:/usr/lib/systemd \
    -v /var/docker-bench-test:/var/docker-bench-test \ 
    -v /etc:/host/etc \
    —-label docker_bench_test \
    gaiaadm/docker-bench-test

The above command, will generate and execute all test cases and will create a TAP test result report in /var/docker-bench-test directory (you need to have write permissions to test result directory).

There are multiple command line flags, that allow you to control docker-bench-test tool. The full list of flags can be printed out if you adding -h at the end of the command line.

docker-bench-test options:

Help documentation for docker-bench-test.sh
Basic usage: docker-bench-test.sh [-c] [-p|-t] [-o path] <test> [<test> ...]
Command line switches are optional. The following switches are recognized.
-c  --Displays number of tests. No further functions are performed.
-g  --Generates all CIS Bats tests without execution. No further functions are performed.
-p  --Show results in pretty format.
-t  --Show results in TAP format. This is the default format.
-r  --Create test results files: tests_<timestamp>.tap in test result folder.
-o  --Specify the test result folder. Default to /var/docker-bench-test/results.
-h  --Displays this help message. No further functions are performed.
Example: docker-bench-test.sh -t -o /var/docker-bench-test/results

The second option to run Docker Bench Test is to clone the project Git repository and run docker-bench-test.sh script manually.

$ git clone https://github.com/gaia-adm/docker-bench-test.git
$ cd docker-bench-test
$ sh docker-bench-test.sh -h

See https://github.com/gaia-adm/docker-bench-test for more details and report any issue you will find there. I will try to fix them fast. I also will keep the project updated with checks from CIS Docker 1.11.0 Benchmark paper.

Related Projects and Relevant Papers

• Original paper CIS Docker 1.11.0 Benchmark
• The Docker Bench for Security tool (scripts)
• Actuary —a new project from the Docker Security team. This tool allows to run only selected subset of CIS Docker Benchmark checks. It can also produce check results in JSON or XML format. It solves two issues, I have with original scripts. I hope one day, it can fully replace Docker Bench Test and save my time :).
• Understanding and Hardening Linux Containers very good and detailed paper by NCC Group
• Docker Security Tools: Audit and Vulnerability Assessment blog post by Toni de la Fuente. Toni periodically updates his post with new tools.