Monitoring Docker With Python & DoMonit

Docker is a powerful tool, however learning how to use it the right way could take a long time especially with the rapidly growing ecosystem of containers which could be confusing, that is why I had the idea to start writing Painless Docker.

Painless Docker Cover

Through this book, the reader will learn and master Docker and a great part of its ecosystem and among other things, the reader will learn how to create his own Docker monitoring system, that’s why I created DoMonit as a poc for this part of the book.

The purpose of this post is to introduce DoMonit: an API wrapper for Docker built to simplify monitoring specific Docker metrics using Python.


Monitoring gives you a visibility about your infrastructure. Running production without monitoring is not recommended at all. 
The same thing is applied to running Docker in production: Monitoring is needed especially if you have a lot of critical applications running containers.

Running Linux systems in production for more than 10 years, I deployed and tested many infrastructure and production monitoring systems.

In my experience, the adoption of managed infrastructure and cloud changed the way we use and manage the infrastructure and I would say that even the nature of the problems are changing, the quality of production is more and more critical and monitoring is becoming more proactive -it is not just the collection and the visualization of several metrics in order to be aware of what’s happening.

In many cases, monitoring could be very specific to a use case or an environment, that’s why I use scripting in these special cases — I almost use Python and sometimes Bash.

In my job and during my experimentations with Docker, Docker Swarm, Micro Services running Docker ..et I needed a simple monitoring utility that I can use in scripts for specific cases without calling the Docker API, that’s why I started Domonit: Deadly Simple Docker Monitoring Wrapper For Docker API

It could be used with Docker API 1.24 and it is compatible with Docker 1.12.x and later, actually works on *nix and other OSs integration will be added soon.

source: https://upload.wikimedia.org/wikipedia/commons/f/fb/Ashdod_Port_Aerial_View.jpg

The Purpose Of Domonit

The purpose is to create python scripts easily in order to monitor all of your Docker containers and collect all of the possible metrics that Docker API offers— few hundreds of metrics and information.

The Github repository of Domonit is : https://github.com/eon01/DoMonit

And the project is a POC, so there are many things to add. It’s easy to understand, so anyone even starting using Docker can contribute.

The Wrapper

The wrapper contains these classes:

domonit/
├── changes.py
├── containers.py
├── errors.py
├── ids.py
├── inspect.py
├── logs.py
├── process.py
└── stats.py

Where :

Containers : List containers

Inspect : Return low-level information on the container id

Ids : Return containers IDs

Logs : Get stdout and stderr logs from the container id

Process : List processes running inside the container id. On Unix systems this is done by running the ps command. This endpoint is not supported on Windows.

Stats : This endpoint returns a live stream of a container’s resource usage statistics.

Usage Example

Create a virtual environment and clone the project

virtualenv domonit
cd domonit
. bin/activate
git clone https://github.com/eon01/DoMonit.git
cd DoMonit
pip install -r requirements.txt
python examples.py

This is the example script:

from domonit.containers import Containers
from domonit.ids import Ids
from domonit.inspect import Inspect
from domonit.logs import Logs
from domonit.process import Process
from domonit.changes import Changes
from domonit.stats import Stats
import json

c = Containers()
i = Ids()
print ("Number of containers is : %s " % (sum(1 for i in i.ids())))
if __name__ == "__main__":
    for c_id in i.ids():
        ins = Inspect(c_id)
sta = Stats(c_id)
proc = Process(c_id, ps_args = "aux")
        # Container name
print ("\n#Container name")
print ins.name()
        # Container id
print ("\n#Container id")
print ins.id()
        # Memory usage
mem_u = sta.usage()
        # Memory limit
mem_l = sta.limit()
        # Memory usage %
print ("\n#Memory usage %")
print int(mem_u)*100/int(mem_l)

        # The number of times that a process of the cgroup triggered a "major fault"
print ("\n#The number of times that a process of the cgroup triggered a major fault")
print sta.pgmajfault()

        # Same output as ps aux in *nix
print("\n#Same output as ps aux in *nix")
print proc.ps()

I had 5 containers running, but for simplicity reasons I will just get the output of just 1 of them:

docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

1e557c8dc5f7 instavote/vote "gunicorn app:app -b " 6 days ago Up 5 hours 80/tcp, 100/tcp vote_webapp_1

the result of the above script is:

Number of containers is : 5 
#Container name
/vote_webapp_1
#Container id
1a29e9652822447a440799306f4edb65003bca9cdea4c56e1e0ba349d5112d3e
#Memory usage %
0.697797903077
#The number of times that a process of the cgroup triggered a major fault
15
#Same output as ps aux in *nix
{u'Processes': [[u'root', u'26636', u'0.0', u'0.2', u'76808', u'16228', u'?', u'Ss', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26773', u'0.0', u'0.2', u'88776', u'19976', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26784', u'0.0', u'0.2', u'88572', u'19800', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26787', u'0.0', u'0.2', u'88568', u'19816', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26793', u'0.0', u'0.2', u'88572', u'19828', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0']], u'Titles': [u'USER', u'PID', u'%CPU', u'%MEM', u'VSZ', u'RSS', u'TTY', u'STAT', u'START', u'TIME', u'COMMAND']}
[..etc..]

Connect Deeper

If you resonated with this article, please subscribe to DevOpsLinks : An Online Community Of Diverse & Passionate DevOps, SysAdmins & Developers From All Over The World.

You can find me on Twitter, Clarity or my blog and you can also check my books: SaltStack For DevOps,The Jumpstart Up & Painless Docker.

If you liked this post, please recommend and share it to your followers.