Reducing the pain of developing apps

10 min readJan 30, 2019

When I write applications I want to spend as much time as possible doing three things:

Define state and the domains of that state
Write logic to change the state
Consume the state in components to produce a UI for the users

When my line of code expresses one of these points I produce actual value for the customer/company. This is where I write the actual application.

In the last few years working with different state management solutions I find myself spending less time writing code covering these three points and more time writing code orchestrating the relationship between them. You might argue that this orchestration creates value in the long run, but there is a threshold. At some point this promise of maintainability, predictability and scalability made my developer experience suck big time.

TLDR; In this article we are going to look at why I think we are spending too much time orchestrating our apps. Then we will look at a solution that minimizes the orchestration required to define, change and consume state, without giving up guarantees of maintainability, predictability and scalability.

Disclaimer. I will be referencing existing tools in a negative way. That does not mean I do not respect the authors or think the tools has no value. On the contrary, that tools mentioned in this article has been very important for the community in general and for my personal learning and productivity.

Orchestrating what now?

To build an application we have three core ingredients. State, logic to change the state and the UI which is produced by consuming the state.

When you spend time with these concepts as a developer you feel productive, you are building the app! As complexity grows we need tools to help us handle the relationship within and between these concepts, and there is no lack of tools. These tools promises maintainability, predictability and scalability. Managing complexity. But even though these tools are here to help, and they do help, they also come with a cost. This is nothing new, but as a developer I have come to a point now where I get really upset by all the time I spend managing the tools, rather than building the app. So let me point out some of these pain points and then tell you how I think they should be solved.

The itches

Typescript

Everyone who knows Typescript also knows that adapting to it is rather painful. What is not always pointed out though is that this has really nothing to do with Typescript itself, it has to do with using tools that was never intended to be used with Typescript. Javascript is a dynamic language which allows a lot of creativity defining APIs, APIs that Typescript simply is unable to express, or the types are not implemented at all. Where I have especially felt pain is with Redux and React component libraries. Redux has as many typing approaches as there are apps, and component libraries does not define their Higher Order Component typing correctly, causing all sorts of weird errors when trying to compose together a component. This is not to say Redux or the component libraries are bad, it is just an unfortunate consequence that creates friction when developing apps.

Flux architecture

When the one way data flow was revitalized by Facebook it got me really excited, cause the concept of having an external state store with logic solved one of the biggest challenges I had developing apps. Now any piece of UI could consume any state and trigger well defined logic to change that state without orchestrating the relationship between the pieces of UI.

That said, even at this day with Redux I think there is still way too much orchestration. Redux`s promise of predictability, by only allowing state changes inside reducers, has a high cost. In addition to defining functions to run logic, you have to define the action types, a switch statement to perform the actual state change, dispatch actions and you ideally have to introduce an immutable state layer. This is a lot of friction to do a simple state change.

You might argue that all this is worth it, but in my experience it is definitely not. There is absolutely no reason to do all these things to get the guarantees of predictability, maintainability and scalability.

Connecting dependencies

I will be bashing a bit on Redux here. Not because I think the library is bad, but it has taken the position as the “holy grail” of state management, despite its high cost. We have already mentioned one cost above, but an other cost is connecting state and actions to components.

For every component that wants access to external state and/or actions needs to explicitly define a function which extracts whatever you want to expose to the component. This in itself is just some additional syntax. Where it becomes a cost is that you have to evaluate what nested level of state you can expose related to how often the component should render. And all this falls back to understanding the concept of reference checking (immutability). If you expose the top level state to a component, the component will render on any state change. If you expose a nested dictionary of posts, the component will render on any change to any post. In certain situations, and those situations always occur in any mid sized app, you have to use shouldComponentUpdate. This is friction and it reduces the developer experience. Defining these connectors and shouldComponentUpdate does not help you build the app, it is primarily you helping Redux and React avoiding unnecessary rendering.

Imperative vs functional

There will always be discussions on where imperative and functional programming shines. To me imperative programming is more straight forward, it is more “how my brain thinks about stuff”. I instruct the computer to make changes to my state, I do not “reduce” or “scan” over state. But that is not my point. My point is that we seem to go for one or the other. If you choose RxJS, everything needs to be RxJS. Or if you choose redux-sagas or redux-loop, everything needs to be defined that way. I think that is wrong. Simple state changes should be done imperatively as it reduces the required syntax and concepts around it. Complex state changes should be done functionally as it reduces the required syntax and concepts around it. Let me explain with an example.

You want to toggle some state. There is absolutely no reason to go through all the hassle of creating action types, implement switch statements and dispatch, or create a stream with RxJS, set up a saga etc. You express this toggling best by simply:

state.stateToToggle = !state.stateToToggle

But then we have something more complex. We want to perform a search. This search should only run when the query is of a certain length and only when the user has stopped typing for 200ms. This would be “a complete mess” implementing imperatively, but functionally you could say something like:

export const search = pipe(
  setQuery,
  filterValidQuery,
  debounce(200),
  getSearchResult
)

Having an API that allows you to move between these two worlds, without locking yourself to one or the other, I believe to be ideal.

Props drilling and events

One of my biggest itches is orchestrating the relationship between components. One thing is composing actual UI made up of multiple components, but when we start orchestrating state and logic between them things gets bad quite quickly in my experience. If you need to pass state multiple components down or you start firing off events which bubbles multiple components up you have created an abstract dependency chain which requires you to jump into multiple files to get your head around it. Another thing is that it becomes fragile. Your actual UI does not necessarily relate to this state/logic dependency chain you have created and as soon as you start moving components around due to changes in the UI itself, things start to break.

Component vs application state

The first reason I got into the open source business of state management tools was an intense itch I had building complex UIs. Whenever I defined state it was in some sense unavailable. The reason was because it was defined within a view/component. As I developed my apps I was constantly refactoring where that state was defined to reach whatever views/components needed access to it. The solution was to move this “application state” outside of the view/component layer. That is not to say component state is not useful, but in my experience you have a better developer experience treating your state as “application state” primarily, cause then by default it will always be available to whatever needs it without any props drilling or refactoring the state to a different view/component. Secondly you use isolated “component state” where that makes sense.

Scratching the itches

All of the before mentioned issues I call “friction”. Things I have to do as a developer that annoys me, breaks my flow or I simply feel is completely unnecessary. The project I am going to tell you about now reduces all the before mentioned friction to a minimum, making application development a more enjoyable experience without disregarding the importance of predictability, maintainability and scalability.

I want to start by giving a shout out to Michel Weststrate and his Mobx project. This project gave a much needed counter weight to Redux and immutability. It also heavily inspired me and my band of open sorcerers to work really hard on choosing the API, and the underlying implementation needed, to lower the friction of app development.

The project never left its working title and is now officially named Overmind. The project has been in development for over 6 months and it could not have seen the light of day any sooner. The reason is that we have been heavily dependent on latest Typescript releases and also getting Proxies into all modern browsers.

Let us now look at how this project reduces the friction of developing apps by revisiting these itches. To get more information about the API and how it works you can visit the official website: https://www.overmindjs.org.

Defining state

Overmind can be used with any of the popular frameworks. It being Vue JS, Angular or React. When you bring Overmind into your project it is encouraged that you primarily define your state in Overmind and you define it as a state tree. That means you do not need any classes or reducers, you just create plain objects which describes the state of your app. You can scale your state into different domains, even lazily load state, and at the end of the day it is all merged together into one state tree.

What we achieve here is to remove concepts like classes, decorators, reducer functions and the likes. You define your state as plainly as possible. This has the benefit of being serializable as well, allowing you to move the state to development tools, rehydrate the state from the server etc.

Changing state

One of the promises of Redux is that the state you define in a reducer can only be changed inside the reducer. That is a great promise and gives predictability. That said, you have to pay for it with action types, dispatching and reducing over state. You do not need a reducer to have predictable state changes. In Overmind you change the state directly with the rest of your logic. There is no action types, dispatching or reducers. Instead, the state changes are scoped to actions. These actions are the only place you are allowed to change the state and the Overmind devtools shows you what actions changes what state.

An additional benefit here is that the actions can safely do asynchronous mutations as well. No need to call a new action or use generators to express asynchronous changes of state, like you do in Mobx.

When your state changes becomes complex you have a natural stepping stone to a functional API exposed by Overmind. You can even reuse your existing actions and compose them with functional operators like filter, map, debounce etc. This allows you to think in simple imperative terms for simple logic and then go functional where it actually gives value. (The search example above is actually Overmind API).

Consuming state

Since Overmind is based on proxies it knows exactly what state your components access. That means there is no reason to map state and actions to props. You just use the state directly and whatever is being used is tracked. That means Overmind optimizes component rendering and it does it a lot better than you would ever do by manually mapping the state. Even though all of this happens implicitly it does not happen out of your control. The Overmind development tool, which is a standalone application, gives you all the information you need. What state is defined, what actions are run, what do they do and what components depends on what state.

I often get the question of, “but I like to explicitly declare the props my component uses”. My theory is that this is a comforting argument as it helps you identify why your component renders, I should know… I have used that argument many times. But now you are relived of thinking about this at all, there is no “will my component render too much” friction.

Typescript

Typescript has evolved a lot since it was first released. It has almost become its own type scripting language, allowing you to express types correctly in a variety of dynamic APIs. One example of this in Overmind is that you can define derived state in your state tree with a function. This function has access to all the state of your app and you return a value based on it. This value is cached and updates when the accessed state changes. When you consume this state in a component though it is typed and accessed as a plain value.

The most important thing though is that Overmind is written in Typescript from the ground up with a focus on the types being part of the actual API. This flips the coin. Typescript is always a help to you in Overmind, it is never a chore which it too often feels with tools not written in Typescript.

Summary

Maybe this article only provoked you. Maybe you are super happy with the state management solution you are using today and it solves building your app beautifully. Good for you! The important thing is to find tools that makes you and your team productive and happy developers. I have only been scratching my own itches and it turns out other developers share those itches and that is why Overmind exists. I hope at least this project has contributed with discussions and questions around what you expect from the tools you use and maybe it even scratches an itch you have, if so head over to https://www.overmindjs.org and learn more about the project!