Scala: Composing Functions Without Saying ‘Monads’

It’s common to view function composition as one of Functional Programming’s best tools for improving code readability and maintainability. Composition helps decoupling concerns and makes your code easier to reuse. The simple case is easy — given two functions where one’s output type is the other’s input type, we can compose a new function that applies the latter to the result of the former. That’s often a better alternative to just writing one big function that performs both transformations — which would be harder to read, would violate the “do one thing” rule-of-thumb for functions, and would couple the two pieces of logic together.

But sometimes, such composition isn’t trivial. When one function’s return type doesn’t match the other function’s input type — composing them is like fitting a square peg in a round hole. In this post, we’ll analyze such a real-world use case we’ve come across in Kenshoo, and discuss how we eventually solved it.

The Problem: Composing Future[Either[T]]

One of our Play-based services contained the following seemingly simple requirement: given some input, apply a series of transformations and return the result. Here’s a simplified version of that requirement:

We have to apply a series of transformations to some input, but each such transformation doesn’t just return a result of the type expected by the next one — it returns that result “wrapped” with an Either (to capture a possible failure of that transformation) and a Future (because the transformation is asynchronous). If it were just one of these wrappers (say, just a Future) — we could have easily used flatMap (or for-comprehension which translates into flatMaps) and be done:

But that’s not the case. We need some kind of “double flatMap”, flattening the Eithers and the Futures at the same time, in order to successfully compose these functions together.

First Attempt: “Follow the Types”

So what would be a simple, minimal, plain-Scala way to implement processRequest? Let’s start with simply “following the types” — applying the first function, then using flatMap and pattern-matching to isolate the case where a result was actually returned, apply the second function to it, and so on. We’ll end up with something like this:

This is obviously hard to read. It also contains some duplication, with the repetitive pattern-matching and handling of the Left case. Lastly, a 4-indentations-deep expression is never a good sign… Just imagine adding a fifth transformation to this beauty.

So how can we improve this?

Second Attempt: Power Tools?

There are some very good libraries to deal with this very problem , or more generally, dealing with “Effects” like Future and Either (e.g. cats). These libraries use the fact that many of these wrappers can be viewed as instances of familiar structures from Category Theory (specifically, Monads), and provide a set of tools to manipulate them (see this excellent presentation for an easy introduction into why and how one can work with Effects).

However, many teams and projects are hesitant when it comes to introducing such tools (along with their notoriously “scary” terminology) into their codebase, and sometimes rightly so. Once you get used to these abstractions, they are extremely powerful and almost seamless. But if you can’t, they can render the codebase impenetrable for you or your teammates.

So in the absence of these “advanced tools” — what can we do?

Third Attempt: DIY

Let’s just look at our very specific problem closely: we would like to end up with the familiar for-comprehension structure, but the series of function applications creates a Future[Either[Future[Either…]]] structure that can’t be “flattened” — the intermingled Futures and Eithers, being two different types, prevent us from using either type’s flatMap function. If we could only convert the “intermingled” type it into a Future[Future[Future[Either[Either[Either[…]]]]] — then each of the two “series of nested wrappers of the same type” can be flattened, which would result in the required Future[Either[…]] structure.

If that’s true, we’ve just reduced the problem to a simpler one — can we implement a “flip” method that turns an Either[Future[Either[..]]] into a Future[Either[Either[..]]]? If we can do that, we can then easily transform that to a Future[Either[..]] (“flattening” the Eithers using Either.joinRight), and then — if we apply that method to each intermediate result — we’ll be able to just “flatmap that sh**”:

We use Either.fold, which takes two functions and applies the first to the left value if it exists, or the second to the right value otherwise. Both functions we use would return a Future[Either[L, Either[L, R]]. Each of these functions just adds wrappers around the existing value. Lastly, we use Either.joinRight, which can be thought of as Either’s “flatten” — turning an Either[L, Either[L, R]] into an Either[L, R] by returning the “first” left or the right.

Let’s see if this helps us implement our processRequest more elegantly:

The end result is clean and simple — once we’ve read and understood the implementation of “flip”. We map each resulting Either using the next transformation, flip it, and then flatMap (using the for-comprehension) into the next one. The last line just uses the fact that the last transformation shares a common Result trait with the Failed type — so instead of returning an Either[Failed, Result] we can just return a Result.

Conclusion

The resulting code isn’t “perfect” (if such a thing even exists). It can be further improved: there are many alternative implementations for the “flip” method, we can replace the method call with instantiation of a wrapping case class that itself implements flatMap and map (thus creating a Monad Transformer — knowingly or not!) and so on. But this got us over the hill, and we can move on to the next one. This version can be easily read and easily extended, and we haven’t had to revisit this since we’ve pushed this into our codebase.

Our takeaway from this: you can experiment with Scala’s powerful composition opportunities, without having to transition yourself and your team to “higher Scala levels” or knowing what Monads are. Occasionally, you’d have to invest a bit of time coming up with such solutions to specific use cases that stand in your way — but these occasions will be both enjoyable and valuable learning experiences. Of course, you can always take the next step, but choosing how deep down the Functional Programming rabbit hole you want to go is totally up to you — there are good solutions scattered all through that journey.