Helidon and JBatch

In this article we will demonstrate how Helidon and JBatch can be used together to execute batch jobs in environments that do not fully support EE environments.

According to the description from the Jakarta Batch (JBatch) official website, the JBatch specification provides the following:

The Jakarta Batch project describes the XML-based job specification language (JSL), Java programming model, and runtime environment for batch applications for the Java platform.

The specification ties together the Java API and the JSL (XML) allowing a job designer to compose a job in XML from Java application artifacts and conveniently parameterize them with values for an individual job. This structure promotes application reuse of artifacts across different jobs.

The specification allows the flexibility for batch jobs to be scheduled or orchestrated in any number of ways, and stops short of defining any APIs or constructs regarding scheduling or orchestration of multiple or repeated jobs.

The usage of JBatch is really vast. Practically everywhere in the enterprise (EE) world there are millions of batch jobs running on millions of servers. The JBatch spec was created to make these types of tasks portable across the enterprise solutions in the Java/Jakarta EE world.

And yes, this is just a specification and not a complete implementation — every vendor has to provide its own implementation, but the specification itself is not standalone. It is very specific and depends heavily on other specs, like JTA and JPA, for example. This means if you want to run JBatch jobs, then you need an Enterprise Server that supports full EE spec.

But since everything is migrating to microservices, where startup time and resource consumption are essential, full support of EE specs is practically impossible. Or is it?

Now with Helidon, for example, even though its built around Netty, and does not support full Jakarta specification, its perfect for the microservices world!

What if we still need to execute batch jobs in environments that do not fully support EE environments? Actually with Helidon there is a now a solution for that!

Let’s create a small example demonstrating this

Helidon is not a full EE container, so how can we use it? For this we will use one of the “so called” standalone (SE) implementations of JBatch. This is not the most common way, but it will work or us. For this example we will use IBM JBatch implementation.

<dependency>            
     <groupId>com.ibm.jbatch</groupId>                 
     <artifactId>com.ibm.jbatch.container</artifactId>              
     <version>1.0.3</version>        
</dependency>

We will also need derby embedded DB, since JPA and JPA are not available by default:

<dependency>            
     <groupId>org.apache.derby</groupId>                 
     <artifactId>derby</artifactId>            
     <version>10.13.1.1</version>        
</dependency>

Now let’s add some jobs. In this demonstration we will use the following: MyItemReader, MyItemProcessor and MyItemWriter … even one MyBatchlet to demonstrate all possible usages of JBatch.

Here we can create a unit of input information:

public class MyInputRecord {
    private int id;
    public MyInputRecord() {
    }
    public MyInputRecord(int id) {
        this.id = id;
    }
    public int getId() {
        return id;
    }
    public void setId(int id) {
        this.id = id;
    }
    @Override
    public String toString() {
        return "MyInputRecord: " + id;
    }
}

The unit of output information MyOutputRecord will look the same.

MyItemReader should look like this:

public class MyItemReader extends AbstractItemReader {
    private final StringTokenizer tokens;
    public MyItemReader() {
        tokens = new StringTokenizer("1,2,3,4,5,6,7,8,9,10", ",");
    }
    @Override
    public MyInputRecord readItem() {
        if (tokens.hasMoreTokens()) {
            return new MyInputRecord(Integer
                  .valueOf(tokens.nextToken()));
        }
        return null;
    }
}

The MyItemProcessor will perform some simple operation:

public class MyItemProcessor implements ItemProcessor {
    @Override
    public MyOutputRecord processItem(Object t) {
        System.out.println("processItem: " + t);
        return (((MyInputRecord) t).getId() % 2 == 0) ? null 
           : new MyOutputRecord(((MyInputRecord) t).getId() * 2);
    }
}

And the MyItemWriter should just print the result:

public class MyItemWriter extends AbstractItemWriter {
    @Override
    public void writeItems(List list) {
        System.out.println("writeItems: " + list);
    }
}

And finally MyBatchlet which simply completes:

public class MyBatchlet extends AbstractBatchlet {
    @Override
    public String process() {
        System.out.println("Running inside a batchlet");
        return "COMPLETED";
    }
}

Now you can put all of this into your job descriptor xml file:

<?xml version="1.0" encoding="UTF-8"?>
<job id="myJob" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/jobXML_1_0.xsd" version="1.0">
    <step id="step1" next="step2">
        <chunk item-count="3">
            <reader ref="io.helidon.jbatch.example.jobs.MyItemReader"/>
            <processor ref="io.helidon.jbatch.example.jobs.MyItemReader"/>
            <writer ref="io.helidon.jbatch.example.jobs.MyItemWriter"/>
        </chunk>    
    </step>
    <step id="step2" >
        <batchlet ref="io.helidon.jbatch.example.jobs.MyBatchlet"/>
    </step>
</job>

As you can see, you will have a two-step job. The first step includes the usage of MyItemReader, MyItemProcessor and MyItemWriter. The last step includes the usage of MyBatchlet.

And this is also where you can see the big difference between EE and standalone implementations — you have to specify the fully qualified names in the `ref` properties, like “io.helidon.jbatch.example.jobs.MyItemReader”, otherwise it will not work.

Ok, now create a small endpoint, which activates the job above:

@Path("/batch")
public class BatchResource {
    private static final JsonBuilderFactory JSON = Json.createBuilderFactory(Collections.emptyMap());
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public JsonObject executeBatch() throws Exception{
        BatchSPIManager batchSPIManager =      
             BatchSPIManager.getInstance();
        batchSPIManager
            .registerPlatformMode(BatchSPIManager.PlatformMode.SE);
        batchSPIManager.registerExecutorServiceProvider(new 
                  HelidonExecutorServiceProvider());
        JobOperator jobOperator = getJobOperator();
        Long executionId = 
                 jobOperator.start("myJob", new Properties();
        JobExecution jobExecution =      
                 jobOperator.getJobExecution(executionId);

        List<StepExecution> stepExecutions =                          
                 jobOperator.getStepExecutions(executionId);
        List<String> executedSteps = new ArrayList<>();
        for (StepExecution stepExecution : stepExecutions) {
            executedSteps.add(stepExecution.getStepName());
        }

        return JSON.createObjectBuilder()
                .add("Steps executed",         Arrays.toString(executedSteps.toArray()))
                .add("Status", jobExecution.getBatchStatus().toString())
                .build();
    }
}

Helidon specifies to JBatch that it should run in Standalone (SE) mode. It will also register the HelidonExecutorServiceProvider which is actually something very small:

public class HelidonExecutorServiceProvider implements ExecutorServiceProvider {
    @Override
    public ExecutorService getExecutorService() {
        return Executors.newFixedThreadPool(2);
    }
}

For our example we need something really small, like a FixedTheadPool with 2 threads. This provider is used to tell our JBatch engine exactly which ExecutorSevice to use.

And you’re done!

Let’s run the code

mvn package
java -jar target/helidon-jbatch-example.jar

Call the endpoint:

curl -X GET http://localhost:8080/batch

We will receive the following log:

processItem: MyInputRecord: 1
processItem: MyInputRecord: 2
processItem: MyInputRecord: 3
writeItems: [MyOutputRecord: 2, MyOutputRecord: 6]
processItem: MyInputRecord: 4
processItem: MyInputRecord: 5
processItem: MyInputRecord: 6
writeItems: [MyOutputRecord: 10]
processItem: MyInputRecord: 7
processItem: MyInputRecord: 8
processItem: MyInputRecord: 9
writeItems: [MyOutputRecord: 14, MyOutputRecord: 18]
processItem: MyInputRecord: 10
Running inside a batchlet

and the following result:

{"Steps executed":"[step1, step2]","Status":"COMPLETED"}

which tells us that the batch job was called and executed successfully.

Conclusion

And now you can see that even though Helidon is not a full EE container, it can still use JBatch in your projects!

If you want to play more with this code, you can find it here ;)

Enjoy!