How to cluster effectively Quartz jobs

Context

A time ago I wrote an article about how to use Quartz with Spring in clustered mode, but because of the active-passive cluster mode of Quartz, where only one node(containers, VM, put your type of infrastructure here) is responsible for handling the incoming “wake-ups” signals of Quartz, we had performance problems in our project, because only one heavy job was consuming all the node resources, resulting in a crash.

In that way, we chose to create a mixed solution using Quartz and Queues to solve this problem.

Photo by Lucian Alexe on Unsplash

The solution

Going directly to the problem, what we chose to do is very simple, instead of one unique service where we embedded the Quartz framework to execute all the business logic of the Job, we chose to split this service into two new services:

• the first one is the Scheduler Service: this service only is responsible to manage the Quartz schedules. When a new “wake-up” occurs, this service sends a notification (message, event, etc) to the Executor Service.
• the second one is the Executor Service: it is responsible to execute the Job itself, doing the business logic, and accessing the needed resources(database, message brokers, caches, etc) to finish the job.