Scatter-Gather: Beyond the Basics

Scatter-Gather is a message router in MuleSoft. You can read about it in detail via Mule docs here. I wrote this blog to share some observations, which are tested below in Runtime.

We know that Scatter-Gather needs at least two routes to be functional. But where would it fail if I included only one route? Will it fail when it builds or when it is getting deployed? Let’s check.

As you can see above, Scatter-Gather will fail while it is being built.

Let’s check out the variable behavior in Scatter-Gather next. Observe the Mule flow below:

You will observe there is a variable called fruits and it is being modified in each route by the addition of one more item. Let’s debug and observe the state variable from the start:

So, we saw that each route received the same variable, and individual routes changes were added to each route. Let see what the final result looks like:

Because I wanted to see all distinct fruits that were added,
I have the final transformation as:

%dw 2.0
 output application/json
 ---
 {
     "content" : payload..payload,
     "fruits" : flatten(vars.fruits) distinctBy $
 }

The final result looks like this:

Now, we observed the behavior in the case of success, but what happens if one route fails? How much can we salvage from the error object? Let’s try by giving 0 as input which will cause a divide by zero error in route 2 or 1 if we start counting from 0. You will observe that when we are inside the Scatter-Gather, the variable behavior remains same as in case of success as the variable is not the error causing component.

Let’s now check the variable value just after the Scatter-Gather.

To our surprise, we have lost most of our fruit when an error occurs.

Let’s see if we can salvage the payload details from the error by doing some transformations on it.

So, we were able to salvage the success payloads, cause for failure, and the Scatter-Gather route number that failed. The transformation I used here is shown below. I know it’s not sophisticated but it works for the purpose of demonstration.

I feel destructive, so let’s fail all the routes.

Let’s compare with the code to verify we are getting the route numbers in correct order.

We have the correct sequence of 100 + “a”, 100/”a”, 10 + “a”, so we are good.

Let’s try another way where we might be able to save our fruits in case of errors.

For these, we wrap each route in a try scope and add an on error continue handler for all try scopes. Let’s observe what happens to the variable this time when we fail all the routes.

So, the changed flow would look like this:

The “errorMessage” inside the on error continue contains:

%dw 2.0
 output application/json
 ---
 {
     errorType: (error.errorType.namespace default "MULE") ++ ":" ++ (error.errorType.identifier default "ANY"),
     errorCause: error.detailedDescription,
     route: 0 //this is varying as per route number
 }

Like before, we have passed “a” as payload and failed all routes. We got the response below from our application:

We got back all of our fruits!
In some cases we might not want to have more fruit, so we can keep figuring out the right error handling approach, as per requirements.

For now, let’s fail one route to verify the behavior with our changed approach.

We have the successful route payload and all fruits in our fruits-variable.

We have seen a lot of error scenarios in a row, so let’s try the success scenario now.

Lets try remove a variable inside Scatter-Gather!

Remove the variable from one route and we will take the fruits without the distinct by function.

As expected, instead of three apples we have two apples, one banana, and one coconut.

What if we use remove variables in all routes, will it completely remove the fruits variable?

It turns out, the fruits variable remains.
If we debug, we will observe that in the individual routes the variable size has become zero inside the Scatter-Gather.

But, when we move out of the Scatter-Gather, we get back the variable fruits with the original values it had before.

To conclude, we can say that the behavior of a variable around Scatter-Gather is definitely interesting.

If you like this article, please leave a clap and follow me for more such blogs.