Dynamic task scheduling with NodeJS and MongoDB
With an unexpected response from my very first blog to task scheduling with Redis here I’m writing another blog on to overcome the same solution with the MongoDB as many of my friends asked for it. Trust me its a very simple and very great solution if you want task scheduling with persistence (Please don’t use setTimeout)
I’m going to show you that how can we use NodeJS with MongoDB for dynamic tasks to be performed by a service on dynamic timings given by the frontend.
So, first thing, why I choose MongoDB for job scheduling? As we all know that MongoDB is a really popular NoSQL Database to be used with NodeJS and NodeJS is now being very popular for the go-to choice for the backend systems, so it can be really easy to adopt this service with your code structure if you are using Node and MongoDB.
So the First thing we’re going to do is define our model first
const Scheduler = new Schema({
taskType: { type: String, enum: ['SMS', 'MAIL'] },
data: { type: Object },
executeOn: { type: Date }
})
Scheduler.index({ executeOn: -1 })So, our model here Defines the task type it can be any enum as per your tasks, data key represent actual data to be sent to the executor of the task later on when we want to execute the task, and executeOn will be used for store the task execution time.
Now we will define the entry point of our system means actually scheduling the task.
it’s very simple as to just insert a document into MongoDB with the details.
const submitTask = async (data) => {
Scheduler
.create(data)
.then(console.log)
.catch(console.log)
}
const executeOn = new Date() // any future date
executeOn.setSeconds(executeOn.getSeconds() + 5)
submitTask({
taskType: 'SMS',
executeOn,
data: {
to: 'BOSS',
message: 'I\'m Still Working'
}
})
So here I have made a function to add data into our collection, and later on, submitting a document into it to send an SMS after 5 sec of the current time to my Boss That I’m still working, here we can take any future date.
Now comes the main part, polling our collection to find the data to be executed, for this query is very simple we need to check the records with executedOn time passed by the current date-time.
const checkSchedulerForTask = async () => {
// check if any task exists
let data = await Scheduler
.findOne({ executeOn: { $lte: new Date() } })
.limit(1)
if (!data) {
// no tasks right now, check after N setTimeout(() => {
checkSchedulerForTask()
}, 1000)
return
} else {
// task exists, send it to execute to other function / queue switch (data.taskType) {
case 'SMS':
sendSMS(data.data)
default:
// do default casing
} // delete that task so it does not repeat and check for the next await Scheduler.deleteOne({ _id: data._id })
checkSchedulerForTask()
}
}
So as we can see the base code it just execute a query on our collection to get any task, If the task exists it can be sent to any function/queue/Pub-Sub/Stream for the execution and that part of your code can execute that task at that specified time.
If no task exists we can wait for some time and again check if any task exists if yes execute it or check back later after some time, that wait time can be configured as per the system needs, here in the example I have set up the wait time for 1 sec only
The last thing to notice is we need to delete that task once it’s been executed so that task will not be repeated.
Here below I have mentioned the sample repo you can check and give your suggestions by commenting down below.
We can use this system as a separate microservice and it is easy to adopt with any code-base, I think the MongoDB solution is much simpler than the Redis solution for the same scenario (Link mentioned below) and far better than using the setTimeout dynamically :)
Please suggest your solution, also don’t forget to Clap Because Clapping is a good habit. 😉
Resources:
