How to Chain Multiple Processors in Spring Batch?
Spring batch is a Spring framework that allows you to work with large amounts of data. If you are familiar with Spring batch’s terminology, you have probably heard of ItemReader, ItemProcessor and ItemWriter that are three key interfaces provided by Spring Batch framework. While ItemReader and ItemWriter interfaces provide bulk reading and writing functionality, ItemProcessor interface is where some business rules are implemented.
When we design our batches, even if performing a single transformation is useful in many scenarios, it can be difficult to perform all business logic on a single processor especially in complex business scenarios. Our approach, in my opinion, should be to design each processor as if this is responsible for one single business rule. To do this, we can use CompositeItemProcessor interface which is provided by Spring Batch framework to manage a list of item processors in a single step.
Let’s make a small example to show how item processors can be chained in a Spring batch.
Our Use Case:
In our sample application, we have input data in csv format that contains some member information. Our batch will process this data according to following business rules and write output data to the file named “outputData.csv”.
a. A member id must be nine characters long.
b. Members whose ids are not starting with 8 shouldn’t be processed and shouldn’t be written in the output file.
c. Persons who have become members in the last six months should be processed and written in the output file.
d. The names and surnames of the members that are satisfying requirement a, b and c should be merged and written as a full name in the output file.
As you notice, above business rules are related to different objectives. Even if those requirements are quite easy to implement in a single processor, it can be hard to manage if additional requirements come to the picture in future. That’s why we will try to separate business logics into several processors.
- Spring Boot(2.6.3)
- Spring Batch
- Java 8
We will create a simple Spring Boot application with Spring Batch dependency. Our pom.xml file is created like below:
We will implement a spring batch job with one main step. The main step contains:
- ItemReader: ItemReader is implemented as FlatItemReader in order to used to read member-list.csv and convert each row to Member object.
- MembershipDateFilter: ItemProcessor for filtering persons who have become members in the last six months.
- MemberNameMerger: ItemProcessor for combining first name and last name
- ItemWriter: FlatItemWriter is used to write processed members to output file.
Since, the main objective of this post is to show how to use multiple processor in spring batch, I don’t give any details about how to configurate FlatItemReader and FlatItemWriter in order to read data from member-list.csv ,and write output data to outputData.csv.
In our sample application, requirement a and b are implemented via MemberIdValidator which implements ItemProcessor interface.
Requirement c is implemented via MembershipDateFilter using ItemProcessor interface.
Our last processor is MemberNameMerger which combines name and surname of a given member as it is stated in requirement d.
Now, we can move on configurating our composite item processor. Here, we encapsulated these three processor within a CompositeItemProcessor that implements the delegate pattern:
Then, we can configurate the main step as follow:
As you can see from configuration, our main step will read data from member-list.csv ten by ten and transform the raw data to our domain object. Then it will send each Member to the compositeItemProcessor. Our custom compositeItemProcessor will call the processors MemberIdValidator, MembershipDateFilter and MemberNameMerger respectively and each processors will accomplish their job within their scope. Finally, utilizing the FlatItemWriter interface, consoleItemWriter will write filtered members to output.csv.
Running the Application:
By running SpringBatchDemoApplication from Intellij as a simple Java application, we can see that the output.csv will created on the classpath.
Then, our batch creates the output file like below:
As you notice, the output file only includes the filtered members with their full names like it is stated in the requirement a, b, c and d.
In this post, I’ve tried to show how to chain multiple item processors in a simple spring batch job. To demonstrate why we should use composite item processors, I created simple batch scenario with multiple different requirements. I believe that, as the complexity of business requirements grows, the concept of composite item processor will aid us in making our batch more flexible and understandable.
You can reached the full source code of this post Github.
Thanks for your time, I hope you enjoyed this post. Let me know if you have any comments or suggestions.