Introducing nion
Today, we’re excited to open source nion, a declarative API data management library built on top of Redux that has solved some big pain points that our frontend team has felt as we scaled our use of Redux. For a little background story, we began porting our front-end from Angular and back-end templates to React three years ago. We chose Redux for our state management needs at that time because it provides a predictable and intuitive data flow pattern for managing shared state across an application. Not only is it elegant and simple, but the developer tooling built around it is top notch. Our application primarily consumes API data adhering to the JSON-API specification, whose benefits closely match GraphQL’s. This has been great as it eliminated many problems that came along to ensure our API is consistent, declarative and agile. However, as our front-end application and development team grew, pain points began to arise.
Problem 1: Unpredictability of API data fetching
Managing network requests and response payloads is hard enough, but doing it using Redux proved to be challenging. This is because Redux, at its core, is a system that relies on pure functions — functions that always return the same output given the same input. By their very nature, network requests are impure because they can return errors and other unforeseen output. In order to accommodate these side effects, the Redux ecosystem has come up with a number of solutions. However, using these libraries are not straightforward due to tricky syntax and complex scenarios, and the patterns for them aren’t quite standardized across the community.
Problem 2: Keeping conventions & patterns consistent
Since Redux is low level, much of an app’s Redux implementation is left up to the developer. While certain practices have become de rigueur, there isn’t anything close to an opinionated framework that developers can use as a guide. Because of this, setting and enforcing code conventions and patterns become extremely important as an application and team grows. We’ve found that as our team grew larger, it became more and more difficult to reinforce usage standards for a framework that inherently doesn’t have consistent conventions and patterns.
Problem 3: Maintaining the source of truth
JSON-API is the specification we chose to use at Patreon. One of the things we like most about it is its ability to define and query data declaratively, much like GraphQL. However, we’ve found that the JSON-API specification is not very friendly to consume. So, because state management can get complex fairly quickly in a large application, the developer is responsible to parse out complex entity relationships, apply entity updates, handle cross-sharing of resources, and so on. This burden significantly slows down the development cycle because it must be replicated nearly every time.
Problem 4: Unscalably sharing resources
The philosophy of Redux includes breaking up apps into smaller, decoupled components that can consume and manipulate global app state through a predictable data flow. Naturally, each resource we add generally ends up with a new set of reducers, actions, and selectors. Sharing selectors especially became a problem when dealing with a graph API where connected component data needs vary across every page, let alone each component. Having to write and compose selectors to parse out the specific API graph data you need has led to extremely brittle code and decreased sharing of code.
Thus, nion
nion is our attempt at addressing the main issues that comes with developing a complex Redux web application able to consume multiple API specifications: REST-compliant, JSON-API spec, etc.. It’s an opinionated solution to the problems we faced building a React web application at Patreon. nion was designed with the specific purpose of making the development experience predictable, clear, and consistent. It always emphasizes the pragmatic solution over the theoretical, and aims to expose the minimum surface area possible to the developer. In short, nion aims to simplify the process of managing API data in a React app.
So what is nion?
nion is a declarative library for managing application data from an API. It’s interface is influenced by Apollo, and emphasizes colocation of components with their data requirements.
nion normalizes the data it manages, which means that changes to data from different places are automatically resolved across the application. This also allows nion to be very performant, with a number of optimizations in place to minimize the number of renders caused by changes to application state.
nion is a library that makes it easy to fetch, update, and manage API data in a Redux store as well as bind it to React components. nion strives to make working with data as flexible, consistent, and predictable when dealing with graph or non-graph based APIs.
And best of all, nion is just Redux under the hood — so the fantastic ecosystem of Redux-centered developer tools and workflows work seamlessly with nion.
Lets look at an example:
How does it work?
The primary interface to nion is used as a decorator high order component pattern which is used to declare what data will be managed by the decorated component and passes in props for managing that data.
Simply pass in a nion declaration object that tells nion what data to manage, and nion automatically handles fetching the data and passing both it and the corresponding request status in as props to the decorated component.
nion significantly reduces the complexity of managing data by both abstracting away all of the logic and code needed to select data and handle requests, and by offering a clear and consistent pattern for doing so. nion offers simple solutions for nearly every type of common application scenario, including component/request lifecycles, updates, multiple requests, pagination, and cross resource entity updates.
The central idea for understanding nion is the dataKey. In the above example, the dataKey is currentUser. The dataKey is the key on the Redux state tree with which nion manages a given resource. A resource is composed of both the underlying data and corresponding network request status for a given piece of application state. Once a dataKey is selected, nion rebuilds the data graph from the normalized data entity store and caches that graph tree for future look-ups into that resource.
Let’s take a look at what the corresponding Redux state tree looks like (after the data has been fetched) to better understand what nion is doing under the hood.
nion manages three internal reducers that handle data fetching and management across the application:
Entities. The entities reducer keeps a normalized map of all given entities in the system, keyed by type and id. This means that all data is kept consistent regardless of where it’s being accessed from. Where there is ever only one source of truth per entity regardless of how many overlapping resources may have requested it.
References. The references reducer maintains a map of dataKeys pointing to the corresponding entities (as an object with type and id fields).
Requests. The requests reducer maintains a map of dataKeys that tracks all network request details around fetching / updating data.
The nion decorator manages selecting all data relevant to a given resource, denormalizing the normalized data and supplying a concise and consistent interface for interacting with the request and data to the wrapped component.
Another benefit of the normalization layer is it keeps the client interface consistent while allowing data consumption from multiple API data sources. For example, we ship nion with 2 api-modules that can consume data from standard REST and JSON-API schemas. This allows applications to consume data from multiple API types as well as be agile by not locking in to a specific API schema. nion also has an extensible layer on top of api-modules on which extensions that can manipulate the request and declaration can be created. This flexibility allows for new functional additions such as JSON-API pagination or API polling.
Open Sourcing & Future
nion has been used at Patreon in production for over a year across the majority of our application, allowing us to work faster, and more reliably, while writing much less code. We have leveraged nion to solve complex pagination, interaction, and state scenarios. While we’re extremely happy to share nion with you, nion is still far from being done! We have a long roadmap of future features. Here’s a small taste for what might be to come:
- Write an api-module for GraphQL
- Pair nion and React’s new Suspense API
- Introduce a <Nion /> dynamic render-prop component
We’re excited to open-source nion and share it with JavaScript community. We welcome your feedback and contributions, and hope as a community we can take nion into the future.
Here’s the repository to get started: https://github.com/Patreon/nion.
P.S. We’re Hiring
Patreon is looking for software engineers of all types and many other roles to join our mission to get creators paid. Please take a look at our job board here: https://www.patreon.com/careers or feel free to reach out directly!
