Custom Sentry Client for more freedom: How to ignore any exception you want
We use Sentry for our error reporting, and we love it. It is simple, gives options for a lot of different use cases, and can easily be integrated with Django, over which our backend is built. We recently fixed (or ignored :P) an issue we had for a long time. This article is about how we did it.
The Problem
Because of a play between Django-Tastypie and Celery-Haystack, our celery workers ended up throwing a lot (5 million in 2 years) of harmless ValueError
exceptions. Due to other priorities and a large amount of effort required, we couldn’t get down to fixing the source of these exceptions. But soon we started reaching our rate-limits for errors on Sentry. We needed a way to make sentry ignore these particular errors.
The Fix
If it was a logged error (like logger.error
), we could simply configure Django’s logging framework appropriately for sentry, but it was an unhandled exception so that would not work.
There are some options for making sentry exclude particular errors:
ignore_exceptions
: This is helpful if you want a particular exception to be ignored, and is a more suitable use-case for ignoring custom Exceptions. But we can’t ignoreValueError
completely since we stand a chance to ignore other genuine Exceptions that must be brought to our notice.exclude_paths
: This should have worked (according to the docs) and I should have just needed to appendcelery_haystack.tasks
, but that didn’t turn out to be the case. I couldn’t make it work in my testing (we haveraven==5.12.0
)
Getting no headway there, and in the interest of time (we didn’t want to spend more time than necessary), we decided to dig into some code and look for a good way to achieve this by overriding Sentry Client.
Looking into the code revealed that skip_error_for_logging
is the perfect method to override for our use. We are taking a list of prefixes that can be defined in Django settings
and testing them against the exception message. If the exception message has one of the specified prefixes, we ignore the error.
Here is our solution:
This provides a lot of control over what exception we want to exclude. And can be extended to check for Exception type
and message
together, use regexes, or any other advanced handling we like.
Result? Our hourly error rate instantly came down. Here is one of the heart-warming screenshots :)
I still have a feeling that exclude_paths
should have worked, but we went ahead with the solution we could figure out the fastest. If someone knows a better way, please let us know :)