Implement Django Watchman Custom Checks in NiceDay Backend
Around the end of last year, we started on a project to rebuild our backend. To build our new backend for our product NiceDay we decided to use Django, which is one of the most popular web frameworks in the Python community.
We used Django along with various Python packages to build our backend. For example, we use Django Watchman to identify whether our RDBMS is up or not. This feature is needed to monitor the status of all services which are required by our Django backend.
Watchman has capability to exposes a status endpoint for your backing services like databases, caches, etc. But its cannot check Redis or MongoDB, so technically we have to build the checkers by ourselves. On the other hand, Django Watchman itself provide us some information to build custom checkers like in this link. Here is sample snippet from documentation:
from watchman.decorators import check@check
def my_check():
return {'x': 1}
But at first glance, I couldn’t really understand what does that means. After I read thoroughly the documentation several times, I got the point from that section that we have to return anything as JSON and has a true value. Whenever a service was down, the JSON response from the custom checks will return improper JSON format. So at that point, we could get information that our service is getting a trouble.
Implementation in our Backend
Finally based on the documentation we build our custom checks to provide information about our service status. For example here is a snippet from our custom checks to assure that Redis is still up on our servers:
from watchman.decorators import check
import redis
from django.conf import settings@check
def check_redis():
r = redis.StrictRedis(
host=settings.CHECK_REDIS_HOST,
port=settings.CHECK_REDIS_PORT,
db=settings.CHECK_REDIS_DB
) r.set('foo', 'bar')
r.ping() return {"redis": {"ok": True}}
In this case we have a data that returned from check_redis(). If its response is not same with check_redis return on Watchman Dashboard, that indicates that our Redis is having some trouble.
Afterward, we try to attach the custom watchman checks to our Django settings.py:
WATCHMAN_CHECKS = (
...,
'util.checks.check_redis',
)Finally, we could see the status of Redis instance via this Watchman Dashboard along with default checkers from Watchman:
{
databases: [
{
default: {
ok: true
}
}
],
redis: {
ok: true
},
storage: {
ok: true
},
caches: [
{
default: {
ok: true
}
}
]
}That’s it, we hope you can start to build your own watchman custom checkers in your backend system.
