NOT containers 101: let’s get more efficient

Abby Fuller
Sep 7, 2018 · 2 min read

Last post, we talked about how to speed up containerized deployments on AWS through configuring load balancer and EC2 settings. Next up, let’s talk about how to get more efficient.


Garbage collection for containers is used to remove things like dangling/untagged images, containers, and volumes. Most orchestration platforms (like ECS/EKS, or Kubernetes) will do some garbage collection for you, but it’s not always enough.

Too many unused images, containers and volumes will:

  • Steal your disk space
  • Wake you up
  • Cost you cash money 💸💸💸💸

Different orchestration tools have different ways of dealing with this. Let’s talk about Amazon’s Elastic Container Service (ECS) first. Here, you can tune the options that you have available for garbage collection. You can fine-tune your parameters by setting ecs-agent options:

ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION
ECS_IMAGE_CLEANUP_INTERVAL
ECS_NUM_IMAGES_DELETE_PER_CYCLE

Don’t know how to set options with the ecs-agent? We’ll get back to that in a minute*.

For Amazon’s Elastic Container Service for Kubernetes (EKS), or Kubernetes, you can do this with kubelet, which also has a cleanup function. You can clean up both images and containers: image collection is based on disk usage, and container cleanup is controlled through kubelet flags (or you can use the defaults):

minimum-container-ttl-durationmaximum-dead-containers-per-containermaximum-dead-containers

Just like with ECS, when you tune these parameters yourself, you need to be careful that you don’t cleanup so aggressively that you lose containers that are useful (for example, a base container that’s already cached).

All of these orchestration tools that we just talked about have defaults for garbage collection. When those defaults aren’t enough, you can

a) tune (like we just talked about),

b) use a 3rd party tool (like docker-gc, from Spotify)

c) all of the above.

At this point, it’s probably worth mentioning that Docker images and containers and volumes aren’t the only way to lose all of your sweet, sweet disk space: everything takes up space! This means logs, and junk you’ve downloaded in user-data, and anything you’ve installed. Some of this is a good use of space! Some of it…is not.

A good example here is log rotate. You can rotate more files, or you can rotate more often. With Ubuntu, you can do that with /etc/logrotate.conf. For example, you could change monthly to daily.

*Configuring the ecs-agent, a belated footnote

For ECS in EC2 mode, you can customize quite a bit through the ecs-agent.

This is good for things like customizing image cleanup, and changing how your instances interact with Docker and AWS (for example, changing Docker flags, or resource usage).

Full list of options is available here. Options can be set in /etc/ecs/ecs.config.

OK, so how do you set options into /etc/ecs/ecs.config? With user-data!


Got a question/comment/tip that I should have covered? Let me know here, or on Twitter: I’m @abbyfuller.

Abby Fuller

Written by

developer relations, agony aunt at Amazon Web Services. Formerly of Airtime, Hailo. tweets @abbyfuller

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade