How to not make a mess managing front-end side-effects

Introduction

Events are great. It’s hard to imagine web programming without the concept of events. They are obviously the foundation of handling any user interaction with the browser but also they are now at the heart of any modern, advanced architecture. Whether it’s backend or frontend, there is usually some form of event-driven architecture going on.

The complexity of patterns used in these different event-driven architectures can vary significantly. From the simplest event notification systems to the most elaborate event sourcing and CQRS ones. Events, however, are what you find at the heart of each such system. What is so special about events? It turns out the fundamental reason why events are so great is because they allow us to practice reactive programming.

Reactive and passive programming

Let’s say you have 2 modules in your system: module “A” and module “B”. If “A” imports a function from “B” and calls it — that is an example of passive programming. In this case there is a dependency from module “A” into module “B”.

Passive model

“A” has to know that “B” exists and the kind of API it has. If, however, “A”, somehow is able to raise an event that “B” listens to and updates itself accordingly — that is an example of reactive programming. Now “A” doesn’t depend on “B”. Pure and simple.

Reactive model

Reactive programming unlocks this inversion of dependencies. We get a system where one part of functionality can respond to an event published by some other part of functionality without being called directly or even being aware of the existence of a module that caused the reaction. That is very powerful.

One of the most important principles of programming is that a module should have only one reason to change and that is a change to its core functionality. With passive programming, a change to module “B” can easily cause module “A” to break. In smaller systems this might not be a big deal but in any sufficiently complex ones it causes massive problems.

While rare in some areas of programming, this concept should sound extremely familiar to any frontend developer. It’s the same idea that is behind any UI. You obviously don’t want your generic button, or checkbox, or any other widget to be aware of any specifics of your code. No, you want it to raise a generic event and then process it accordingly and independently.

With the growing complexity of the frontend systems, this concept has been making its way from just UI into more and more layers of our applications.

Managing application state

The problem of state management is one of the main sources of complexity in any application. State management can encompass many layers of the application. State may change in response to user interactions, to API requests, some background running tasks, to changing network conditions, etc. This problem is definitely complex enough for us to be looking for help from reactive programming. Luckily in the last several years there has been a growing number of frontend frameworks and libraries that tackle this problem. Many of them use the principles of reactive programming.

Redux

We, at Crunchyroll Web, use Redux. Redux is great. What’s so great about it? Well, it is based on events and lets us react to things in a decoupled way, and that, as we have already established, is great.

When any user interaction happens, you can fire up an event (or action in Redux terminology) and then you can decide how you handle this in a function that computes the next state, called a reducer. All this happens in a completely decoupled way. Perfect. However, by default the ability to respond to events is reserved for reducers only. Reducers are intended to be pure functions used only to compute a new state.

Let’s say we load something from the server, we raise an event, then that data is transformed in the reducer and finally gets stored in our state. What if we want to go the other way? What if we want to send a request in response to an event? In fact what we are looking for is an ability to run arbitrary side-effects in response to any event.

Asynchrony

How can Redux help us with our problem? Well, we’re out of luck as our use-case is not even handled by default in the Redux architecture. The solution offered in the documentation is to install a separate special “thunk” middleware. This middleware would extend the capabilities of Redux to be able to handle asynchronous actions. Sounds pretty abstract. Let’s look at a concrete example.

Code Example

Let’s say you have a users feature. In that feature you have to load the users. No problem, you create a thunk for that in your actions file:

// users/actions.js
function loadUsers() {
  return (_, dispatch, {API}) => {
    return API.getUsers()
      .then(data => dispatch(usersLoaded(data)))
      .catch(error => dispatch(usersLoadFailed(error)));
  };
}

Simple enough. Then a new requirement comes along and now you have to report an error to some service when users fail to load. No problem just add that in:

// users/actions.js

function loadUsers() {
  return (_, dispatch, {API}) => {
    return API.getUsers()
      .then(data => dispatch(usersLoaded(data)))
      .catch(error => {
        dispatch(usersLoadFailed(error))
        crashAnalytics.reportError(error))
      });
  };
}

How difficult is that? Do we even need another solution?

Problems with Thunks

While this may not seem like a big deal, what you’ve just done is coupled your users feature to your error reporting service. If its API changes, you will have to update the users feature. The users feature is the one responsible for reporting the error. When we write tests for the loadUsers function we will have to test that it calls the crashAnalytics service. What if the logic for the error reporting grows? What if you need to track it differently depending on the error code? What if another requirement comes along and now you also have to display a flash message displaying the error information to the user? And then another requirement for analytics tracking on the successful loading of users. All that logic will be added to the loadUsers function, making it a messy hairball of intermingled functionality.

How can we separate those concerns? Well, you should already know the answer by now. Reactive programming. Thunks use a passive programming model and cannot be triggered by dispatching a plain action. And the passive programming model is at the origin of all the problems mentioned above.

So, to recap: we love Redux because of its reactivity but the reactive solution provided by Redux cannot be asynchronous and asynchronous solution provided by Redux cannot be reactive.

Available solutions

Since we are obviously not the first ones to encounter such problems there are already established solutions in the Redux eco-system. Two of the most popular ones are: “redux-saga” and “redux-observable”. These are definitely powerful libraries and as such they bring their own set of complexities. “Redux-observable” is based on RxJS reactive programming library and it can certainly be overwhelming for anyone who hasn’t had any experience with it. “Redux-saga” is based on standard ES generators but still brings a lot of custom functionality and also their interface seems a bit clunky for our seemingly simple use-case.

Build vs. Bring

On our team when we encounter a problem our first course of action is to explore how we can solve it ourselves. We obviously stay aware of the already existing solutions but very often they tend to solve many more problems than is necessary. Usually at least testing what it would take to code our own solution is generally a good idea. If that proves too cumbersome or time-consuming we can always switch to a 3rd party solution.

What is it that we want?

To evaluate the potential complexity of implementing an in-house solution let’s first dream-up an interface. Back to our loadUsers example. Instead of making loadUsers a thunk we make it an “effect” that responds to the LOAD_USERS action:

// users/effects.js
const loadUsers = ({getState, API}) => [
  LOAD_USERS,
  action =>
    API.getUsers()
      .then(usersLoaded)
      .catch(usersLoadFailed)
]

The loadUsers effect runs every time LOAD_USERS action fires, runs the API.getUsers() effect and maps to an action that needs to be dispatched. In addition to the action itself, we also would like to inject other dependencies, like getState, API, analytics service and others, so we make effects functions that can accept those.

Writing a custom Redux middleware to get this code to actually work turned out to be pretty simple so we kept rolling with it.

Improving on initial version

After converting several of our thunks to effects we began thinking on how to improve our interface. For a 2.0 version we wanted to implement just a few basic operators, similar to “redux-observable”. This way we would be able to turn an effect like this:

const removeWatchlistItem = ({getState, API}) => [
  REMOVE_WATCHLIST_ITEM_REQUESTED,
  action => {
    const state = getState()
    const isUserAuthenticated = isUserAuthenticated(state)
    if (!isUserAuthenticated) {
      return
    }
    const watchlistItemId = getWatchlistItemId(state)
    return API.deleteAccountWatchlistItem({id})
      .then(() => removeFromWatchlistSuccess({id}))
      .catch(error => removeFromWatchlistError({id, error})
  },
]

Into the following:

const removeWatchlistItem = ({getState, API}) =>
  compose(
    ofType(REMOVE_WATCHLIST_ITEM_REQUESTED),
    map(getState),
    filter(isUserAuthenticated),
    map(getWatchlistItemId),
    flatMap(id =>
      API.deleteAccountWatchlistItem({id})
        .then(() => removeFromWatchlistSuccess({id}))
        .catch(error => removeFromWatchlistError({id, error}))
    )
  )

The great thing about our operators (in contrast with RxJS operators) was their simplicity. They were just a few lines of code each and any member of our team could understand how they worked. Also even after migrating a lot of code to effects we only needed about 10 of them, as opposed to close to 100 that are included in the RxJS library.

After coming up with a composable interface for operators this was an obvious upgrade. This turned out to be our final interface. Writing additional operators is very rarely needed but if it is — it’s a very simple process.

What we got

Let’s get back to our example with loading users, see how it’s done now and what benefits this new approach brings.

// users/effects.js
const loadUsers = ({getState, API) => [
  LOAD_USERS,
  action =>
    API.getUsers()
      .then(usersLoaded)
      .catch(usersLoadFailed)
]

Now let’s integrate our analytics service. The code responsible for the loading of the users remains completely untouched. We simply add an analytics effect:

// analytics/effects.js
const reportError = ({analytics}) => compose(
  ofType(USERS_LOAD_FAILED),
  map(prop(‘payload’)),
  end(analytics.reportError)
)

Good. If our reporting logic grew, loadUsers would remain unaware of that. Displaying a flash message would simply be another effect:

const apiErrorNotification = () => compose(
  ofType(USERS_LOAD_FAILED),
  map(() => flashMessage(‘generic_api_error’))
)

Benefits

As you can see from the example all features are now almost entirely decoupled from each other. They are also small and testable. New additional requirements usually don’t introduce bloat to any existing effects but rather create new, smaller, decoupled effects of their own. They can live in entirely different directories and moreover we can now completely remove the error analytics reporting and none of our other code would break.

Adoption

We were initially worried this new approach may be confusing to some developers. However, effects have proven to be very straight-forward and easy to use. Since their introduction they have spread like wildfire throughout our codebase and are a de-facto standard for handling pretty much any asynchronous interaction between modules. As of now, no one on our team writes thunks anymore. In fact, we went from having hundreds of them to eliminating ~99% and have even considered dropping the “redux-thunk” dependency altogether.

Future plans

The most important thing that happened with this explosion of effects is a shift in our team’s mindset. We began to think reactively. When modules begin to be too tightly coupled or some function call is placed in an inappropriate place — we notice it.

We have since found many use-cases for the effects other than replacing thunks and continue our refactoring journey.

For example there is still some application startup logic and analytics tracking that is not handled via effects. We are actively refactoring all that code as well and should soon live in the “effects paradise”.