Reading files and sending file content to an API using WSO2 Enterprise Integrator

Smooks Mediator

WSO2 Enterprise Integrator is a fully open-source product that helps to integrate and connecting different heterogeneous applications systems. It has the capabilities of Business Process Management, Enterprise Service Bus, Messaging queues, Analytics and Tooling.

WSO2 Integration Studio is the tooling kit for WSO2 Enterprise Integrator, It has an eclipse based plugin as well as a visual studio code based plugin. These plugins make you a graphical environment so that you can design your APIs, Workflows visullay by dragging and dropping components from the palette to the working area. As well as if you are more familiar with XML configurations you can go with the configuration-based approach.

While implementing those integration-related use cases I came across a scenario where the user wants to read a CSV file from a specified folder location, read it and send its details to SOAP-based backend.

If you think about this in a code-based approach there is quite a lot to do, but with WSO2 Enterprise Integrator and its tooling support, it is just a matter of dragging and dropping components from the pallet and configuring their properties correctly.

Implementation

Create an ESB solution project using the WSO2 Integration studio. Create an endpoint for SOAP API and create a file processing proxy within that project.

1. Creating a SOAP endpoint.

Right-click on your ESB solution project, goto New > Endpoint and click on Create a new endpoint, Select address endpoint as endpoint type and provide a suitable name for your endpoint.

Configure your endpoint details as below.

Address Endpoint

Remember to specify the correct SOAP version of your SOAP endpoint under the Format attribute.

So this is the endpoint which we are going to send data retrieved from the CSV file. Whenever you create an endpoint it will be listed under defined endpoints in your palette. So this will also become a drag and drop component. So you can drag and drop this endpoint to your API or Proxy service easily.

2. Creating a file processer proxy.

Right-click on your ESB solution project. Goto New > Proxy service and click on Create a new proxy service, keep proxy service type as custom proxy and enable VFS under select transports, then give a suitable name for the proxy service.

Proxy service type, VFS means Virtual File Service transport type, which is a transport type used in WSO2 ESB to process files in the specified source directory. It will also delete or move files after processing as specified.

So when you specify the transport type as VFS, you also need to specify details like input folder location, action to be taken after files processed ( delete or move), If files need to be moved after processing where to move, action to be taken if file processing failed, If failed files need to be moved where to move, what is the poll interval to check the input folder and so on.

Now open your proxy.xml file in source view to add above mentioned properties. You can add them as following examples, inside <proxy> element

3. Implementing proxy

Now design your proxy service as below by dragging and dropping components. Here “soapEndpoint” is the endpoint I created in step 1.

proxy service in graphical view

proxy service in source view

Here I also have added some property mediators and header mediators to specify SOAP properties like SOAP action, content type.

Here you will see, there is a smooks mediator added. this is the mediator which will convert processed file data to XML. To specify these conversion details we need to add a separate local entry and then specify that local entry name in smooks mediator properties. As you will see, I have specified it as “smooks-config1”.

4. Creating a local entry

Now right-click on your ESB solution project, goto New > Local entry and click on Create a new local entry, Select in-lined XML entry as local entry type and specify a suitable name for the local entry. ( In our case it is “smooks-config1”). Now copy and paste the following content to the value section.

Here, Under param name fields, I have column names for CSV columns. these will be converted to XML attributes under “recordElements” with record element name payment_data. Since the CSV file contains multiple rows, the root element will contain multiple record elements. and the root element will have the name of add records.

CSV file to be processed

So following shows the SOAP request, after smooks did its job.

Summary

Smooks is a powerful framework for manipulating, processing, transferring XML. As WSO2 Enterprise Integrator supports smooks, using smooks mediator you can do powerful message, file processing using WSO2 Enterprise Integrator.

Thank You