Joys of being a hacker — Using Iron MQ for web application monitoring
When you think of queue the immediate thing which comes in your mind is producer — consumer problem. Iron MQ is a massively scalable queue and when you first look at it you say Ah! this is perfect! No hassle of maintaining your own queue. Highly available and super scalable. Now you only need to put task from anywhere in the world and have your workers work on the task at their own pace. Well that’s how most people use queues.
Looking beyond the normal queues, Iron MQ offers some brilliant features. eg. push queues. You can have any http endpoint as a subscriber. So you push a message on the queue and the message will be POSTed to your endpoint. You can add a lot of subscribers to the queue and have the queue post to them in unicast or multicast. Then there is this brilliant concept of error queues. If your endpoint times-out or returns an error code (5xx) then the message can magically be pushed onto an error queue. You can even specify delay or eta which means I can say this particular message should be pushed onto the queue only after t seconds.
Necessity is the mother of all inventions
I was looking around for some hosted monitoring solutions for a web application. Most good ones were at least 10$ a month. It was a bit of a waste for what i really wanted. Then suddenly i had that imaginary light bulb shining bright on my head and i told myself why can’t i use Iron MQ (free subscription) for my monitoring? Basically what do i really want? Web server is up or not? Is the redis cache, mongo db and rabbit mq available and working fine from the web server?
Getting dirty
I quickly wrote 4 handlers.
- NodeJS — Return immediately.
- Mongo — Read a value from DB.
- Redis — Write a random value on the cache.
- RabbitMQ /Celery— Put a task on the queue and wait for worker to process it and reply.
In all the case if there is error return status code 500 otherwise return 200.
On Iron MQ i created a push queue and added the above for handlers as subscribers. Then in a loop i push messages on the queue with different delays.
for i in range(24*60): q.post({“body” : “iRock”, “delay”: i*60})
This will create 1440 messages each with progressive delay so basically every minute one message will be introduced onto the queue for next 24 hours. I have setup the error queue also. As soon as message is on the queue all our four links are POSTed to and if any of them times out or returns error then that error is moved to the error queue. All you got to do is keep check on the error queue. You have got yourself pretty robust monitoring solution for free!
Going further — Notifications (SMS/E-mail/Phone)
I have made the error queue a push queue also and pointed it to start a Iron Worker. The worker is very powerful and you can do almost anything with it. Just now all it does is emails me then waits for 5 minutes and sends me a SMS and then again wait for another 5 minutes and then calls me on the Phone. I use Twilio for SMS/Phone and Mandrill for emails. It’s just couple of lines of code. In the meantime it keeps waiting on the queue and if there are any more messages then it arranges them according to type and puts in Iron Cache for analysis later. The idea behind time wait is that if i see the notification i will logon to Iron and kill the task ;-) A little crude but serves the purpose for the time being.
To automate the creation of monitoring queue I have written a worker which creates messages for 7 days and is run every week with the Iron task scheduler.
While i don’t claim this to be the ultimate monitoring tool but it is rock solid, scalable, free and easy to customise. And it servers the purpose for most developers.
