Use Logentries with Docker Compose application

The 11th item of 12-Factor application tells us to handle logs as stream of events and recommend to send each of them in a centralized location.

In the context of a Docker Compose application this setup can be done in a couple of minutes.

Note: before setting this up, please make sure that all the services of your application send their logs to stdout / sdterr.

A lot of logging solutions exist out there, open source solutions that needs to be installed and operated on premise, commercial solution that handle everything for you as soon as the log are correctly redirected.

I stumbled upon Logentries during DockerCon Europe in Barcelone last year (great event btw !) and have started to test this solution a couple of weeks ago. I really wanted to go for a managed solution.

Our Compose application

To keep thing simple but a little bit interesting, let’s say we have the following Docker Compose application.

Basically, a web front-end backed up by a Node.js api using MongoDB as datastore. The web and api services use RabbitMQ as key-value store to handle sessions. The whole application is accessible through a reverse-proxy that forward requests to web or api based on the url.

None of those services handle their own logs, they all send them to stdout / stderr.

Here comes Logentries

Right after having created an account on the Logentries platform (free plan does not require any credit card, and offers some useful functionalities to start analyzing the logs), we can select one method to send our logs.

As this screenshot shows, a lot of methods are available ! The one we are interested in is the little whale at the bottom right :) Clicking on Moby brings us to the next configuration step.

After giving a name for our new log set and validating, we are explained which command to run to send logs towards Logentries platform.

docker run -v /var/run/docker.sock:/var/run/docker.sock logentries/docker-logentries -t 82855f8b-5341–34d5–3e54–12b612c26efa -j

Basically, this command tells us to run a container based on logentries/docker-logentries image. The Docker daemon’s socket of the host is mounted into the container so this guy can see all the events happening on the host and send them to Logentries platform. The application is identified by the token generated during log set creation.

Note: several other options can be provided while running the container.

Adding logentries service in docker-compose file

We’ve just seen there is only need to run a container on the Docker host to send logs towards Logentries web service. Let’s just add this new container in our Compose file.

Defining this new service in docker-compose.yml, enables to get all the other services events.

Visualizing our logs

With this easy setup, we can access our logs and performs some filtering operations on them (regexp, …). More complex operations, live tail, S3 storage, … are available in premium plans.

In the above screenshot we can see logs coming from different services in our staging environement (https, api, web, …).

Summary

I’ve just started to test Logentries integration in our Docker Compose application. I really enjoyed the very easy setup in a Docker Compose context.

The free plan is great to filter logs, we’ll probably go for the next plan soon.