Golang logging and unix logrotate
The marriage made in heaven
Golang log package
Golang log package which comes with logger interface provides everything we want to deal with logs. It capables to write to all the standard devices, custom files or any destination that support the io.Writer interface. By default log packages does not support log rotations. To do log rotations we can use unix logrotate package.
Unix logrotate
Logrotate helps to manage all of our log files. It can periodically read, minimize, back up, creates new log files, and basically anything you may ever want to do with them. According to the documentation logrotate allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.
Scenario
Below is the scenario that I have used logrotate package to daily rotate golang application log files,
I have golang based crypto application/microservice which runs on docker. I’m using mongodb as the certificate store. golang log package using for logging. To daily rotate the logs I’m using unix logrotate package.
Following are the steps that I have followed to achieve daily log rotation in my application.
1. Log configuration
In here I’m configuring log file and other log flags to write the log messages with timestamp and lineno. All the log messages goes to zwitch.log file which reside on config.dotLogs directory(which comes via envirounment variable) inside docker container. I’m mounting that directory in to host directory as a docker volume. Then I can access these logs from host machine.
2. Logging
I can initialize the log configuration from my main function and reuse it from all over the application.
In here I’m initializing log configs, then I’m setting up mongodb connection. All the error message and other message logging in to zwitch.log file.
3. Run dockers
Following is the docker compose file that can use to up and run the services. I have two services zwitch and mongodb
zwitch container logs goes to the app/.logs directory inside the container. I’m mapping app/.logs directory to /private/var/services/zwitch/logs directory inside the host machine. So zwitch.log file can be found in /private/var/services/zwitch/logs/zwitch.log location in my host machine. You can up and run these services via docker-compose up -d command.
4. Logrotate config
I have created logrotate file /etc/logrotate.d/zwitch with the log rotate configuration of the zwitch service. /etc/logrotate.d is the location that we need to put custom logrotate configs. This directory contains logrotate config files of various services.
Following is my logrotate config file for zwitch service
daily flag use to rotate logs daily basis
copytruncte flag truncate the original log file to zero size in place after creating a copy, instead of moving the old log file and optionally creating a new one
compress flag use to compress the log files when rotating
dateext flag use to include the date to rotated log file name
dateformat flag gives the format of the date
dateyesterday flag use to add the previous date to the rotated log file name. Otherwise it will add rotation executed date, which is today
5. Test logrotate
To test the logrotate config, we can forcefully run logrotate config we have specified. In this example I have used sudo logrotate -f -v /etc/logrotate.d/zwitch command to run the logrotate with zwitch config file. This command will execute logrotate config and create the rotated log file with yesterdays name.
