Tutorial: collecting and analyzing Docker container logs with Sumo Logic (for free)
When it comes to powerful log analytics options for Docker, there are many commercial options out there, but they can be quite expensive. I’m going to share one which offers a free tier without compromising on the features.
Docker has a built-in logging driver called
json-file . Container logs are formatted as JSON and written to a text file per container. You use the
docker logs command to view the logs as plain text. The only filtering options available are
until. You may pipe the output to
grep to do keyword search, same as with plain text file logs.
This is good enough when you want to view logs for a single container and debug issues ad-hock, right now. For anything more complex (e.g. parsing, filtering and sorting) you‘ll want a real log collection and analytics tool.
There are multiple options out there. Some are paid. Some are free and open source, but DIY. There is one, though, which combines the power and convenience of a commercial SaaS platform with a free entry level plan for small projects and hobbyists — Sumo Logic.
Sumo Logic offers a free plan with 0.5GB/day (15GB/mo) log data ingestion. If you are smart about what goes into your container logs and how often, you’ll be able to stretch the data allowance and get the power of a professional SaaS log collection and analytics platform for free.
Let’s dive in!
Sign-up for a free trial with Sumo Logic. You will have to use your “work” email address. Consumer mailboxes like Gmail won’t work.
Under Administration > Security > Access Keys click the + icon in the top right corner to add a new key:
Pick the Access Key Label and click Generate Key. Copy and store your keys somewhere secure. You will use these when starting the Sumo Logic Docker Collector container.
Start the collector container on your Docker host
$ docker run -d -v /var/run/docker.sock:/var/run/docker.sock --name=sumo-logic-collector --restart=always sumologic/collector:latest <AccessID> <AccessKey>
<AccessKey> with the values you recorded previously.
Give the collector a minute to initialize, then go to Manage Data > Collection to confirm it shows up in your account and logs data.
There will be two data sources streamed by the collector container:
- Docker-stats — CPU/memory/network/etc. container stats
- Docker-logs — actual container logs
If the collector does not show up in Sumo Logic, then check the collector container logs for clues:
$ docker logs sumologic-collectorRunning SumoLogic Collector...wrapper | --> Wrapper Started as Console
wrapper | Java Service Wrapper Standard Edition 64-bit 3.5.13
wrapper | Copyright (C) 1999-2011 Tanuki Software, Ltd. All Rights Reserved.
wrapper | http://wrapper.tanukisoftware.com
wrapper | Licensed to Sumo Logic Inc. for Collector
wrapper | Launching a JVM...
jvm 1 | WrapperManager: Initializing...
jvm 1 | . . . . . . . . .
jvm 1 | .+'|=|`+. .+'| |`+. .+'|=|`+.=|`+. .+'|=|`+.
jvm 1 | | | `+.| | | | | | | `+ | `+ | | | | |
jvm 1 | | | . | | | | | | | | | | | | | |
jvm 1 | `+.|=|`+. | | | | | | | | | | | | | |
jvm 1 | . | | | | | | | | | | | | | | | |
jvm 1 | |`+. | | | | | | | | | | | | | | | |
jvm 1 | `+.|=|.+' `+.|=|.+' `+.| |.| |+' `+.|=|.+'
jvm 1 | Sumo Logic Collector Version 19.209-26
jvm 1 | Sumo Logic Build Hash fa2afe3
jvm 1 | current folder:/opt/SumoCollector
jvm 1 | * See /opt/SumoCollector/./logs for more details.
jvm 1 | * Connecting to https://collectors.sumologic.com.
jvm 1 | * ERROR: Registration failed: Your Sumo Logic credentials could not be verified. Make sure the token or accessKey/ID is valid and your user account has permissions to manage Collectors. (error key: collectors.unauthorized)
jvm 1 | Collector exiting...
wrapper | <-- Wrapper Stopped
UTC timezone is used by default for logs timestamps. You may want to adjust that in the settings (Edit link) as necessary.
See the official sumologic-collector-docker repo for additional configuration options and documentation.
Docker stats dashboard in Sumo Logic
Let’s start with some nice dashboards and graphs available out of the box.
Sumo Logic has a “Docker App” available in the App Catalog. Go ahead and add it to your Library.
Set the Source Category for logs to docker
Once the app is installed and some data is collected, you will see some nice graphs pulled for the containers stats.
Searching and analyzing container logs
Now let’s see how we can get to the actual container logs in Sumo Logic.
Create a new Log Search using the “+ New” button in the top right (you can also use the alt+s keyboard shortcut for this)
To only view container logs use the
_source=Docker-logs filter. You can also narrow the search down by collector name, source host, etc. Sumo Logic will automatically suggest filter options and available values.
To view logs for a specific container — add the
_sourcename filter, e.g.
AND _sourcename = "plex-server"
To search for a specific log message or keyword, either add it manually in the search query as
AND <keyword> filter or highlight it and use the options from the dropdown.
Easy! Now we are only viewing the specific messages we are interested in.
Once we got down to the instances of the messages we are interested in, we may want to see what else was captured in the logs around that time. Sumo Logic makes this extremely easy as well. Click on any of the filters under a message and select Surrounding Messages and a timeframe:
Disclaimer: I’m not affiliated in any way with Sumo Logic and did not receive any form of compensation from them or anyone else to write this article. I was looking for a solution for my own needs and found Sumo to fit it very nicely.
I hope you enjoyed reading and found this tutorial useful.
Clap all 50 times if so! This helps others discover content on Medium.