Monit(or) your Apps

To monitor critical processes, get alerts (emails/chat notifications), to restart automatically a process that crashed for any reason, we all need a tool.

Some of us use SaaS like New Relic, Nagios, or Scoutapp.
Some of us prefer solutions like God or Bluepill.

There is another one.

Today i’m gonna talk about another one: Monit.
It’s opensource (AGPL), simple, stable and fast.

You need root rights to install it:

apt-get install monit

You can find it then in /etc/monit/. Here are 3 things:

• a monitrc file: The config of monit is here.
• a conf.d directory: At the end of monitrc, this directory is included, so all specific config will be added here.
• a monitrc.d directory: ready-made config files for main process you often wanna monitor.

monitrc

The config file will let you specify how often you wanna check processes, where the log files are handled, some basic checks on the system, files and programs are also added in comments.

Around the lines 118–123 we can add a web interface to see our checked processes being green or not, plus others things:

set httpd port 2812 and
  use address localhost 
  allow localhost # allow localhost to connect to the server and
  allow admin:monit # require user ‘admin’ with password ‘monit’

The first line set the webserver on the port 2812 (default port for monit)
The second line says we only accepts connection from localhost, which is ok here because we’ll use Nginx as proxy.
The third and forth line are already explained.

Save the file and type :

sudo monit -t

If everything is configured properly it should answer:

Control file syntax OK

Then just reload monit:

sudo monit reload

Nginx

In your main server bloc config add the following location :

location /monit/ {
  rewrite ^/monit/(.*) /$1 break;
  proxy_ignore_client_abort on;
  proxy_pass http://localhost:2812; 
  proxy_redirect http://localhost:2812 /monit; 
}

No magic here, it is a copy/paste from the documentation.

Now go to yourdomain.example.com, you should see something like this:

Monit Web UI.

Add checks in conf.d

Thanks to existing examples in /etc/monit/monitrc.d you can just cp this files in config.d/ for them to be included in checks, they will also appear in the Web UI. Try:

sudo cp /etc/monit/monitrc.d/nginx /etc/monit/conf.d/nginx
sudo monit reload

And reload your WebApp. Nginx processes and files should now appear.

Here is an example of non-existing check I added:
/etc/monit/config.d/redis

check process redis-server
  with pidfile “/var/run/redis/redis-server.pid”
  start program = “/etc/init.d/redis-server start”
  stop program = “/etc/init.d/redis-server stop”
  if 2 restarts within 3 cycles then timeout
  if totalmem > 500 Mb then alert
  if children > 255 for 5 cycles then stop
  if cpu usage > 95% for 3 cycles then restart
  if failed host 127.0.0.1 port 6379 then restart
  if 5 restarts within 5 cycles then timeout

The same goes for sidekiq, unicorn, postgresql, …

Restart your app on deploy

You might need to add some task in your deploy script to restart some processes (like unicorn and sidekiq).
In Capistrano I added this:

namespace :deploy do
  task :restart do
    on roles(:app), in: :sequence do
     sudo “monit restart unicorn”
     sudo “monit restart sidekiq”
    end
  end
end
after 'deploy:publishing', 'deploy:restart'

There is difference between Monit and M/Monit.

M/Monit expand on Monit’s capabilities and provides monitoring and management of all your Monit enabled hosts.

M/Monit is licenced and on pay-once model. It is really cheap for the service it gives.

Have fun !

If I forgot anything, or got wrong or more complicated than needed, please tell me ! I’m always eager to learn a better way :)