Email Scheduling Application with JobRunr

Published in

eDreams ODIGEO

9 min readNov 3, 2021

JobRunr

In this post we are going to create an Email Scheduling Application using Java 11, Java SpringBoot and JobRunr [1] to schedule emails (jobs).

What is JobRunr?

JobRunr is an amazing open and free library that allows performing background processes on the JVM. It provides a technology to execute fire-and-forget, delayed, scheduled and recurring jobs in Java applications. Indeed, those jobs only need to be defined as Java 8 Lambda.

Furthermore, the jobs are persistently stored and JobRunr provides a great level of abstraction of the storage due the jobs could be stored in different databases. Some examples are defined as follows:

RDBMS such as MySQL, Oracle, Postgres, MariaDB, SQL Server, DB2 and SQLite.
NoSQL like Redis, ElasticSearch or MongoDB.

The execution guarantees that a single scheduler is performed due to an optimal locking and it provides a distributed and cluster-friendly capability. It also supports and intensive usage of CPU and I/O, long-running and short-running jobs.

When is JobRunr a great option?

JobRunr could be a great solution for those scenarios that implies a background execution or some kind of scheduling.

A great examples provided by the JobRunr’s team [2] as follows:

When a REST API needs to response to client immediately while long-running job is performing in the background
Mass notifications/newsletters
Heavy operations and its result
Batch import
Creation of some kind of resources
Firing off web hooks
Video/Image processing
Recurring automated executions (application maintenance, reports, etc)
Database operations (database maintenance, updating, etc)

Basically, it could fit in any operation that must be executed in the background. In addition, it could perfectly fit an event driven architecture due to scheduling features.

How it works?

1) The jobScheduler enqueues and/or schedules (it could be recurrently) the background job. It is declared as a Java Lamba’s which is decomposed and stored into the storage provider as Json.

Figure 1: UML class diagram of the jobScheduler scope when it enqueues or schedules. Serializing the lambda and storing the job.

2) JobRunr immediately returns the jobId to the client avoiding blocking its business logic.

Figure 2: The jobScheduler UML flow/activity diagram.

3) One or more background job servers read and poll enqueued and scheduled jobs from the storage provider and process them

Figure 3: UML class diagram of the backgroundJobServer scope reading the jobs stored in the Database.

The BackgroundJobServer is initialised with the JobRunrConfiguration. It is created with a ServerZooKeeper and JobZooKeeper.

Figure 4: A constructor of BackgroundJobServer from the jobRunr project [2].

When the backgroundJobServer starts, both Zookeepers (ServerZooKeeper and JobZooKeeper) are started. Both implement the Runnable interface and they are added in the executor service ScheduledThreadPoolJobRunrExecutor.

Figure 5: The booting of the zooKeepers in BackgroundJobServer [2].

Moreover, they are going to be executed recurrently every configured period. Otherwise, it uses the default configuration defined by jobRunr.

Figure 6: The default values defined in BackgroundJobServerConfiguration [2].

The ServerZooKeeper is checking the status of the background servers with a heart beat in order to check if it is alive, basically, it sends a signal. Unless some servers are not responding, it removes those background servers that did not respond with the heart beat from the database. Another action that the serverZooKeeper applies is defining the master background server. It can be seen in the following picture that it is an extraction of the source code:

Figure 7: ServerZooKeeper’s logic [2].

On the other hand, the JobZooKeeper updates the jobs that have been executed, checks if there are some recurring, scheduled or some jobs that should be deleted and enqueues them. Finally, it communicates to the backgroundServer to add all the enqueued jobs in the JobRunrExecutor .

Figure 8: JobZooKeeper’s logic [2].

4) When all of that is finished, it starts the process again.

Figure 9: Schema of JobRunr architecture.

Furthermore, it can be used for light processing for an application or it can be scaled horizontally and allows adding more background job servers to support a peak of processing. Indeed, JobRunr provides a great level of loading over all the servers and is also fault-tolerant due that if a job fails it has its own retry policy and tries 10-times with a smart back-off policy. So, basically, it will try to perform the job, and if it fails, will wait some time, then try to execute again until reaching the limit of tries.

How can background jobs be monitored?

JobRunr incorporates a built-in dashboard where it provides helpful insights into your background jobs. Basically, it allows monitoring and controlling your background, scheduled and recurring jobs in your application. In addition, a detailed view on any exception that occurred during the execution can be seen and checked.

Figure 10: Extraction of jobRunr dashboard.

Moreover, jobRunr also includes a REST Api for consulting and managing their resources:

Get the jobs and their status
Delete a job
Requeue a job
Get the background servers and their status
Get issues

The definition of that REST Api server can be seen in the following picture:

Figure 11: REST Api JobRunr definition [2].

Disadvantages or limitations

Even being a great library and technology to use in some applications, JobRunr has some disadvantages or limitations.

The main disadvantage is:

The jobs cannot be updatable. They must be deleted and regenerated.

Also, it requires generating an extra relationship between the entities and the jobs. In that tutorial the email transaction with the job. Indeed, it provokes a need for synchronisation between those entities and their TTL (time to live) because the job can be deleted after a successful execution.

Figure 12: Relationship between the entity and the job.

Mailer Scheduler

A great email scheduling softwares will allow you to organize your email list and deliver the proper message at the desired moment to the right prospect.

As we have explained we are going to use Spring boot, JobRunr to schedule the jobs and JavaMail Api to send emails using Java. Also, important libraries that we are using such as MapStruct for mapping Objects [3] and the dependencies required by JobRunr [4] (ASM [5] and Jackson [6].

Moreover, that application is going to be ready to use as storage a RDBMS or NoSQL database, more specifically MySQL and Redis.

JobRunr configuration

Firstly, we need to configure the project with the spring boot dependency and the jobRunr dependency as it is shown in the following picture:

<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>org.jobrunr</groupId><artifactId>jobrunr-spring-boot-starter</artifactId><version>${jobrunr.version}</version></dependency>

Then, the required dependencies by JobRunr should be added (ASM and Jackson).

<dependency><groupId>org.ow2.asm</groupId><artifactId>asm-util</artifactId><version>${asm.version}</version></dependency><dependency><groupId>com.fasterxml.jackson.core</groupId><artifactId>jackson-databind</artifactId><version>${jackson.version}</version></dependency>

Regarding the storage, the dependency of the desired option must be added. For instance:

MySQL

<dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId></dependency>

Redis

<dependency><groupId>javax.mail</groupId><artifactId>javax.mail-api</artifactId><version>${javax.mail.api.version}</version></dependency>

Let’s consider if we choose a jpa based solution, the Spring Data JPA module should be integrated.

<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-jpa</artifactId></dependency>

JavaMail Api dependencies should be added as well in order to send emails in JAVA.

<dependency><groupId>javax.mail</groupId><artifactId>javax.mail-api</artifactId><version>${javax.mail.api.version}</version></dependency><dependency><groupId>com.sun.mail</groupId><artifactId>javax.mail</artifactId><version>${javax.mail.api.version}</version></dependency>

JobRunr could be configured programmatically and easily in the application.properties of Spring. Due to its similarity to a tutorial, in that case an hybrid approach is going to be chosen. It is going to be initialized programmatically but some configurations are going to be located in the properties.

The JobScheduler is going to be injected using the following code:

@Beanpublic JobScheduler initJobRunr(DataSource dataSource, JobActivator jobActivator) {return JobRunr.configure().useJobActivator(jobActivator).useStorageProvider(SqlStorageProviderFactory.using(dataSource)).useBackgroundJobServer().useDashboard().initialize().getJobScheduler();}

As it can be seen in the above code, the JobRunr is set up with the StorageProvider (in that example an SQL), the backGroundJobServer and the Dashboard to monitorize all of it.

In the properties, some other configurations can be placed. The properties definition and class can be found in the JobRunrProperties class from the library.

For instance, if you want to configure something extra for the BackgroundJobServer, it can be checked and added looking at the attributes in the class.

Figure 13: BackgroundJobServer properties definition in JobRunrProperties[7].

So, looking at the above code enabled, workerCount and pollIntervalInSeconds can be configured. And it is actually what has been done in that project among others as it can be seen in the following code.

org.jobrunr.database.skip_create=falseorg.jobrunr.database.table_prefix=mailer_schedulerorg.jobrunr.job-scheduler.enabled=trueorg.jobrunr.background-job-server.enabled=trueorg.jobrunr.background-job-server.worker_count=8org.jobrunr.background-job-server.poll_interval_in_seconds=60org.jobrunr.background-job-server.delete_succeeded_jobs_after=36org.jobrunr.background-job-server.permanently_delete_deleted_jobs_after=72

Most of them are self explanatory themselves.

Java Mail Api configuration

The Java Mail Api needs a smtp configuration. In that tutorial, the Gmail smtp server is going to be used for sending those scheduled emails.

The configuration is added in the spring property file as it can be seen in the following part of code.

Figure 14: Java Mail Api properties.

Furthermore, and just for tutorial purposes the password for the smtp server is sent through the request.

@JsonIgnoreProperties(ignoreUnknown = true)public class EmailRequest implements Serializable {private String subject;private String sender;private String password;private String receiver;private String text;

Furthermore, and just for tutorial purposes the password for the smtp server is sent through the request. That password is an application password which should be configured in the gmail account.

Figure 15: Set up application password in gmail.

Implementation

First of all, the best practices in that implementation must be followed:

Use names instead of verbs
Naming of resource should be plural as representative of the collection
Nest the resource to represent relation and hierarchy
Error handling management with appropriate HTTP Status code.
Versioning

One of the big common errors is to use verbs in our APIs to indicate the action of the endpoint. However, the resource should be used with a plural name and the HTTP method must indicate the action to perform. The most common and important HTTP methods are as follows:

GET retrieves data from a specific resource. In that tutorial jobs and transactions emails.
POST creates a resource, mostly storing it in the database. Jobs and the transactions
DELETE removes the resource.

Another common error is not using the HTTP Response status codes properly. As a REST is a representation state, it must indicate the status of those requests.

2xx (Success category)
4xx (Client error Category)
5xx (Server error category

The REST Api implementation is defined like that:

@RestController@RequestMapping(value = “/mailer-scheduler/v1”,produces = MailerSchedulerConstants.JSON_CONTENT_TYPE)public class MailerSchedulerRestServiceController@GetMapping(value = “/ping”)public String ping()@PostMapping(value = “/jobs/mails”)public ResponseEntity<EmailTransaction> postJobMail(@RequestParam(required = false, name = “scheduleTo”) String scheduleTo, @RequestBody EmailRequest emailRequest)@DeleteMapping(value = “/jobs/mails/{id}”)public ResponseEntity<Void> getMailJob(@PathVariable(“id”) Integer id)@GetMapping(value = “/mails/{id}”)public ResponseEntity<EmailTransaction> getMail(@PathVariable(“id”) Integer id)

Moreover, into the business logic, the relationship between the email transaction and the job should be generated in order to persist and link those two entities.

@Overridepublic EmailTransaction addEmailJob(LocalDateTime scheduleTime, Email email) {String jobId = jobScheduler.schedule(scheduleTime, () -> mailerService.sendEmail(email)).asUUID().toString();return emailTransactionRepository.save(EmailTransaction.builder().setJobId(jobId).setSubject(email.getSubject()).setSender(email.getSender()).setReceiver(email.getReceiver()).setEmail(email.toString()).setScheduleTime(scheduleTime).build());}

In order to persist that relationship which has been generated in the database, the spring boot jpa library is used [8].

Conclusions

JobRunr is a great choice as an open and free library that allows performing background, schedule and recurring processes/jobs with great performance and scalability.
It provides a great monitoring system of those tasks due its dashboard and REST Api.
Jobs are not updatable, so a custom workaround should be implemented. For instance, removing and regenerating the job with the new information.
it requires generating an extra relationship between the entities and the jobs and its synchronization.