[WSO2 ESB] Iterate and Aggregate JSON Requests.

Split — Aggregate Integration Pattern.

In Enterprise World, Split and Aggregate is one of the vital Integration pattern. With WSO2 ESB, we have few ways to achieve this integration pattern. One of it is using Iterate and Aggregate Mediator.

Using Iterate mediator, we will be sending out the request from the client to the multiple endpoints and using aggregate we will be combining the responses from the back end and replying back to the client as a single response. In this blog post, I will be explaining how to split a JSON array which is coming from the client and send each element of the array to the back end and how to aggregate/combine the response from the back end and sent it to the client.

Request From the Client

[{
    "Name": "Ram",
    "Age": 26
},
{
    "Name": "Niro",
    "Age": 25
},
{
    "Name": "Niro Ram",
    "Age": 27
}]

Expected Request by the Back end

{
    "Name": "Ram",
    "Age": 26
}

Response From the Back end

{
    "Name": "Niro Ram",
    "Status": "Accepted"
}

Expected Response by the Client

[{
    "Name": "Ram",
    "Status": "Accepted"
},
{
    "Name": "Niro",
    "Status": "Rejected"
},
{
    "Name": "Niro Ram",
    "Status": "Accepted"
}]

Lets see, how we can implement this using the ESB Iterate and Aggregate mediators.

First, we need to split the response ans send each and every response to the back end. For this purpose, we will be using an Iterate mediator.

Iterate Mediator

<iterate id="NIRO" expression="//jsonArray/jsonElement" sequential="true">
    <target>
        <sequence>
            <header name="Content-Type" scope="transport" value="application/json" />
            <call>
                <endpoint>
                    <address uri="http://www.mocky.io/v2/5b77c37e2e00002b00864bb0" />
                </endpoint>
            </call>
        </sequence>
    </target>
</iterate>

Then after sending the requests to the back end we need to aggregate the responses from the back end, for this purpose we will be using an Aggregate Mediator.

Aggregate Mediator

<aggregate id="NIRO">
    <completeCondition>
        <messageCount min="-1" max="-1" />
    </completeCondition>
    <onComplete xmlns:ns="http://org.apache.synapse/xsd" expression="$body/*[1]" enclosingElementProperty="Aggregated_Responses">
        <send />
    </onComplete>
</aggregate>

This full fills our requirement mentioned above. You may add Log mediators in between if you need to monitor the changes. Following is a sample proxy which contains all of the above mediators tested in Enterprise Integrator 6.2.0.

Sample Proxy

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="splitAggregate" transports="http https" startOnLoad="true">
    <description />
    <target>
        <inSequence>
            <iterate id="NIRO" expression="//jsonArray/jsonElement" sequential="true">
                <target>
                    <sequence>
                        <header name="Content-Type" scope="transport" value="application/json" />
                        <call>
                            <endpoint>
                                <address uri="http://www.mocky.io/v2/5b77c37e2e00002b00864bb0" />
                            </endpoint>
                        </call>
                    </sequence>
                </target>
            </iterate>
            <property name="Aggregated_Responses" scope="default">
                <jsonObject />
            </property>
            <aggregate id="NIRO">
                <completeCondition>
                    <messageCount min="-1" max="-1" />
                </completeCondition>
                <onComplete xmlns:ns="http://org.apache.synapse/xsd" expression="$body/*[1]" enclosingElementProperty="Aggregated_Responses">
                    <send />
                </onComplete>
            </aggregate>
        </inSequence>
        <outSequence />
    </target>
</proxy>

Few points to discuss here,

Note the property “Aggregated_Responses”, this contains the tag within which we are going to enclose the responses. Since there are going to be multiple responses, we need a parent tag to enclose them. In our case, since it is JSON, I have added <jsonObject/> tag.

Note the iterate id and aggregate id being same. The requests and responses are identified based on them, hence we need to have them identical.

Note the onComplete condition in aggregate mediator. Basically if we put -1 it will wait till all requests are executed successfully. We may modify it based on our requirements.

Please refer the following WSO2 documentations for more details.

Enterprise+Integration+Patterns

Iterate+Mediator

Aggregate+Mediator

Note : With WSO2 EI 6.5.0, native JSON support has been introduced in Iterate mediator and you can use JSONPath expressions to iterate over JSON payloads. For more info please refer https://medium.com/@arunans23/iterate-over-json-payload-with-wso2-enterprise-integrator-8ccb9cdd2c70

Thank you for reading !!!