Docker disk usage analysis
At Uniplaces we run our applications in Docker containers and we have a local environment used to test services against each other in the development stage.
Before dockerizing everything, our playground was based on a virtual machine with a significant amount of disk space was allocated to it. In most cases, it was hard, if not impossible, to reduce the disk and memory footprint.
By having the playground based on a network where we can attach containers to; each application can have its own docker-compose configuration that encapsulates how each container should be identified on the network.
Before starting
The first command is docker psand is used to list all the containers showing the status, the image used to build it and with the command was launched with:
docker ps is focused on the connectivity, showing what ports are exposed inside the network and how the container is exposed outside the network.
The second one is docker volume lsand is used to list the volumes:
The third one is docker volume lsand is used to list images:
With these commands, we have enough information to see the images available in the docker env, which can be useful for us to start a new container.
Disk usage analysis
So let’s start digging the system, the goal is to maintain the disk of our machine clean while understanding what resources are been consumed. To start this journey let’s invoke the docker system df command:
At the moment there are only images, seven in total, occupying almost 3GB and all of them are removable as there’s nothing referring to them.
To get some more details, such as what images, containers and volumes exist, the size and other specific metadata per docker resource, add the verbose flag:
Let’s start some services and see how things change.
There are now five containers running with two volumes, none of this space is reclaimable while the reclaimable space for images went down to 38%.
Being more verbose:
Working with volumes
Two of the five running containers have an attached volume, to know what container is using a volume let’s run:
This volume is used by the redis-dev container.
Combining the two listing commands with some shell scripting is possible to list all the containers and their usage:
Here, for each volume is extracted the name that is then used in the filter query to select the containers which are using it.
Applying the dangling filter to the docker volume ls command the output will show only the unused volumes.
docker volume ls -f dangling=trueThese are the images that would be deleted running docker volume prune
Working with containers
When the goal is to clean up all the docker environment the first resources to be removed must be the containers as they lock all the rest (images, volumes, networks).
docker ps comes with a flag to list all the containers and combined with the quiet flag will output only ids of all the containers, running or not. It’s now easy to remove all the containers in one line:
docker rm -f $(docker ps -q -a)Working with images
The listing “system df with running containers verbose” shows that the image 200cbd58d198 has not been used, let’s remove it:
For the sake of example, let’s get the image back; an image can be pulled as
A docker image is made of layers, pulling an image means pulling the layers that compose the image itself and removing an image means removing the layers that’s why there are several lines when removing or pulling an image.
Sometimes, it can be useful to investigate how an image has been built, in other words, its history:
To better view the layers let’s combine the no truncate flag with the format option
It can be useful to list the images ordered by their size to be easier to evaluate what image is using more disk. Docker with some shell scripting to help can accomplish this by:
docker image ls —-format ‘{{.Size}}\t{{.Repository}}:{{.Tag}}\t{{.ID}}’ | sort -hr | column -tFinally, with the same kind of command used to delete containers, images can be deleted with:
docker rmi -f $(docker images -a -q)Besides the force flag, this command will not delete an image if it’s used by a container or if it has dependent child images.
Prune the system
After all is good to remember a powerful command that deletes all the unused data:
docker system pruneThis will also delete unused networks, a topic that wasn’t spoken in this article because docker networks don’t consume the disk space.
Conclusions
This set of commands is enough to maintain the system clean removing what is occupying the disk unnecessarily. Finally, to perform any kind of analysis docker commands have the inspect subcommand, that combined with the format option become a powerful tool for investigating any docker resource.
