On D3, React, and a little bit of Flux
When I first started exploring D3 and React together, I realized I wanted to attempt it differently from the other approaches I had read about. I had just read through the React tutorials and had started on an article (Nicolas Hery’s Integrating D3.js visualizations in a React app) when inspiration struck. I had also heard murmurs that wrapping D3 charts in React made transitions hard, and I knew I had to at least give solving it a try. When I mentioned this to Elijah, he encouraged (demanded?) me to write it down in a blog post.
So I want to tell you about the problems that got us here, how I decided to use D3 and React together, and how Flux factored into the decision. And because I’m a bit obsessive, I’ve created an example expense app (which got a little out of control) for demonstration purposes. You can find the app here, and the source code here. To play with it, download the app, and npm start.
The problems that got us here
I work on the front-end team at a security company called Illumio, on a project called Illumination. Illumination is a visual tool for users to gain insight into their environments, and to help build rules that secure that environment. It’s a really exciting project that uses D3.js heavily for drawing the visualizations, and Backbone.js for managing the data. But as Illumination became more complex, with more parts and user actions that could change the visualizations, we started to see more and more problems emerge.
At the core, the problems boiled down to the following two categories:
- We were trying to “intelligently” update the graph with D3’s enter-update-exit pattern.
- We were firing Backbone events for all data changes, creating an unsustainable spaghetti ball of events that was hard to debug.
D3 and React
D3’s enter-update-exit pattern gives a lot of control to the developer: we manage when an element is added to the screen and when it’s removed, as well as how it’s updated. That’s great when the updates are simple, but it gets overwhelming when there are a lot of elements to keep track of, and the elements to update vary from one user action to another (which was what we were facing in Illumination). Even worse, we were trying to be “smart” about it; in order to prevent DOM churn, we would calculate which elements required updates, and soon we were getting overwhelmed because we couldn’t keep the changes straight in our heads anymore. We were manually diffing the DOM tree.
When I first heard about React two years ago, its promise of intelligent updates with the virtual DOM sounded exactly like D3’s enter-update-exit. So I didn’t think twice about it until last summer, when we started to consider different Javascript libraries in our UI re-architecture. And the more I read into it, the more it made sense in our situation: let React do all the migraine-inducing heavy lifting to figure out what to enter and exit, and have D3 take care of the updates.
Entering and Exiting with React
React has a concept of Dynamic Children, which works very similarly to D3’s data binding. It encourages us to pass in a key per child to keep track of the order of children (much like the key function we pass into d3.data), and uses it to calculate what should be added and removed when the data changes.
Consider this set of expenses:
This is what rendering just the structure (two rectangles and a text element) would look like with D3:
var graph = d3.select(‘svg’).append(‘g’)
.classed(‘graph’, true);var expenses = graph.selectAll(‘g.expense’);var entered = expenses
.data(expensesData, (expense) => expense.id) // use id as key
.enter().append(‘g’)
.classed(‘expense’, true);
entered.append(‘rect’)
.classed(‘expenseRect’, true);
entered.append(‘rect’)
.classed(‘textBG’, true);
entered.append(‘text’);expenses.exit().remove();
And this is the equivalent with React:
class ExpenseComponent extends React.Component {
render() {
return (
<g className=”expense”>
<rect className=”expenseRect” />
<rect className=”textBG” />
<text />
</g>
);
}
}class GraphComponent extends React.Component {
render() {
var expenses = expensesData && expensesData.map((expense) => {
// go through all data and return components keyed by id
return (<ExpenseComponent key={expense.id} data={expense} />);
}); return (
<svg>
<g className=”graph”>
{expenses}
</g>
</svg>
);
}
(From components/Expense.jsx and components/Graph.jsx)
The React code looks like a lot more overhead, but it does two great things for us:
- It forces us to componentize the elements, so it’s easier to reuse (we were doing this in our old D3 code too, but React makes it explicit).
- It comes pretty close to mirroring the structure of the DOM, so it’s easier to keep track of what the components look like.
The second point also means that I don’t ever have to think about entering and exiting again. I used to have a headache whenever we had to do something like this:
entered.select(‘rect’)
.classed(‘textBG’, true)
.classed(‘hidden’, (d) => !d.name);entered.select(‘text’)
.classed(‘hidden’, (d) => !d.name);
It’s a pretty contrived and ugly example, but we’ve had to do things along these lines in Illumination, where we show and hide parts of the component depending on the data. With React, I can tell it to draw only the elements I need, when I need it, in a beautifully straightforward manner:
class ExpenseComponent extends React.Component {
render() {
return (
<g className=”expense”>
<rect className=”expenseRect” />
{this.props.data.name && (<rect className=”textBG” />)}
{this.props.data.name && (<text />)}
</g>
);
}
}Updating and Transitioning with D3
Now that we have the structure of the components, let’s fill in the attributes. I’m a huge fan of D3’s transitions (let’s be real, I’m a huge fan of everything D3), and I wasn’t willing to lose that in our move to React.
So let’s have D3 manage updating the attributes:
var ExpenseVisualization = {};ExpenseVisualization.enter = (selection) => {
selection.select(‘rect.expenseRect’)
.attr(‘x’, (d) => -d.size / 2)
.attr(‘y’, (d) => -d.size / 2)
// …
.attr(‘stroke-width’, 0); selection.select(‘rect.textBG’)
.attr(‘opacity’, 0)
// …
.attr(‘fill’, ‘#fafafa’); selection.select(‘text’)
// …
.attr(‘opacity’, 0)
.text((d) => d.name);
selection
.attr(‘transform’, (d) => ‘translate(‘ + d.x + ‘,’ + d.y + ‘)’);
}ExpenseVisualization.update = (selection) => { selection.select(‘rect.expenseRect’)
.transition().duration(duration)
.attr(‘width’, (d) => d.size)
.attr(‘height’, (d) => d.size)
// … animate box in; selection.select(‘text’)
// … position text element; selection.select(‘rect.textBG’)
// … position text background; selection
.transition().duration(duration)
.attr(‘transform’, (d) => ‘translate(‘ + d.x + ‘,’ + d.y + ‘)’);
}
(From visualizations/Expense.js)
In the React component, we call the enter code from componentDidMount, and the update code from componentDidUpdate. This way, as soon as the element is inserted into the DOM, we can use D3 to set its starting attributes, and when any part of the data changes, we can have D3 transition the element to its next set of attributes.
class ExpenseComponent extends React.Component {
componentDidMount() {
// wrap element in d3
this.d3Node = d3.select(this.getDOMNode());
this.d3Node.datum(this.props.data)
.call(ExpenseVisualization.enter);
},
shouldComponentUpdate(nextProps) {
if (nextProps.data.update) {
// use d3 to update component
this.d3Node.datum(nextProps.data)
.call(ExpenseVisualization.update);
return false;
}
return true;
},
componentDidUpate() {
this.d3Node.datum(this.props.data)
.call(ExpenseVisualization.update);
},
render() {
// …
}
}(From components/Expense.jsx)
And here’s the very important part: D3 should never manipulate what React is already keeping track of.
The virtual DOM keeps track of the structure we tell them to, so if at any point of the React lifecycle we change the DOM without its knowledge, it throws nasty bugs (and I mean the kind of bugs that are nasty because they’re subtle, and take half a day to track down). (This Stackoverflow answer covers the danger quite well.)
So the pattern helps a lot in keeping a clear line of ownership between the two, where React manages the structure, and D3 manages the attributes. This way, D3 can transition elements — update its fill color, update its position, update its size — all without conflicting with what React knows.
More on transitioning with D3
You might have noticed in the previous section that the expenses are transitioning independently of each other; they all transition in at the same time. That’s because the transitions are called by the expenses themselves, each with no knowledge of its siblings.
One of the things I find really fun about d3 transitions is that we can stagger the animations with transition.delay:
It turns out that staggering the transitions doesn’t take too much more work with the code we already have. The key difference is that whereas before, the transition happened when an expense component finished its own React update, now they need to happen when all of the expense components finish their React update. Luckily, we have a parent Graph component whose own componentDidUpdate doesn’t get called until all its children are finished updating, and we can orchestrate the transitions then.
So the changes we need to make are quite minimal:
ExpenseVisualization.update = (selection) => {
selection
.transition().delay((d, i) => d.order * duration)
.duration(duration)
.attr(‘transform’, (d) => ‘translate(‘ + d.x + ‘,’ + d.y + ‘)’);
}GraphVisualization.update = (selection) => {
selection.selectAll(‘.expense’)
.call(ExpenseVisualization.update);
}class GraphComponent extends React.Component {
componentDidMount() {
this.d3Node = d3.select(this.getDOMNode());
},
componentDidUpdate() {
this.d3Node.call(GraphVisualization.update);
}
}
(From visualizations/Expense.js, visualizations/Graph.js and components/Graph.jsx)
A caveat
A note with this pattern is that we can’t animate elements transitioning out. Even though React gives us a componentWillUnmount hook, it seems to be called mere milliseconds before the element is removed and doesn’t give us enough time to complete an animation. This problem can be solved by using React’s TransitionGroup, which waits to remove an element from the DOM until an animation has completed; Jeremy Stucki has a great example of this on his bl.ocks.
A little bit of Flux
Before with Backbone and D3, I used to calculate display attributes (size, fill, stroke, etc.) of an element in its model and store it in that same model. But a user might edit an element, which triggers a model change, which triggers a recalculation of the display attributes. That by itself was fine, but when a display attribute depended on the state of multiple models or even collections, the resulting change events and race conditions made debugging a nightmare.
So when I first read about Flux, it clicked right away; its pattern of unidirectional data flow made so much sense for solving the problem of spaghetti events. We didn’t need to reason through views that listened to models, models that could be mutated by views, and all of the ways user actions could ripple through the app. All we needed to do was tailor the Dispatcher to our own needs, and follow the patterns for stores and actions. We followed the Flux flow for every user action, updating the stores and recalculating what should be shown on the screen.
There are three main actions a user can take in the expense app: adding categories and expenses, dragging expenses to categorize it, and editing an expense. A category’s display attributes are highly dependent on its expenses; adding an expense to a category draws a link between them and increases its size. Editing an expense’s amount changes the size of all of the categories it’s linked to. Deleting either an expense or a category deletes the links between them. When any of these actions occur while a user is inspecting the category or expense, the details in the left panel also change.
To keep all of those actions from becoming a fountain of bugs, we delay the display attribute calculations as far down the Flux cycle as possible (something my coworker insisted on when we moved to React and Flux). This maintains the stores — CategoryStore, ExpenseStore, GraphStore — as the sources of truth, and keeps them from becoming polluted by the calculations.
In the expense app, we calculate the graph data in Graph.jsx, and pass the data down to Category.jsx, Link.jsx, and Expense.jsx via props. The graph data is recalculated anytime GraphStore or SelectionStore emits a change, or anytime ExpenseApp.jsx — which listens to changes in CategoryStore and ExpenseStore — passes down new props:
class GraphComponent extends React.Component {
getInitialState() {
return {
categories: [],
expenses: [],
links: []
}
},
componentDidMount() {
GraphStore.addChangeListener(this._onChange);
SelectionStore.addChangeListener(this._onChange);
},
componentWillReceiveProps(nextProps) {
this._onChange(nextProps);
},
componentWillUnMount() {
GraphStore.removeChangeListener(this._onChange);
SelectionStore.removeChangeListener(this._onChange);
},
// use the next props to calculate and set the next state
_onChange(props, width, height) {
props = props || this.props; var selection = SelectionStore.getSelection();
var categories = AppCalculationUtils.calculateCategories(props.data.expenses);
var expenses = AppCalculationUtils.calculateExpenses(props.data.expenses);
var links = AppCalculationUtils.calculateLinks(categories, expenses);
// calculate some more rendering things
AppCalculationUtils.calculateSizes(categories);
AppCalculationUtils.highlightSelections(selection, categories, expenses);
// calculate their positions
AppCalculationUtils.positionExpenses(expenses);
AppCalculationUtils.positionGraph(categories, expenses, links); var state = {categories, expenses, links, dates, width, height};
AppCalculationUtils.calculateUpdate(this.state, state);
this.setState(state);
},
render() {
var links = this.state.links.map((link) => {
var key = link.source.id + ‘,’ + link.target.id;
return (<LinkComponent key={key} data={link} />);
});
var categories = this.state.categories.map((category) => {
return (<CategoryComponent key={category.id} data={category} />);
});
var expenses = this.state.expenses.map((expense) => {
return (<ExpenseComponent key={expense.id} data={expense} />);
}); return (
<svg>
<g className=”graph”>
{links}
{categories}
{expenses}
</g>
</svg>
);
}
}
(A subset of components/Graph.jsx)
A note on updating
So in the React and D3 section, I said to let React handle entering and exiting the elements, and D3 handle updating the attributes. There’s one problem with that: if an attribute that React doesn’t keep track of changes, React wouldn’t know to update that component. The change won’t get reflected in the DOM because React will never reach the componentDidUpdate part of the lifecycle, and the corresponding D3 code will never be called.
To get around that, we want to go through the data and indicate which ones have changed since the last time we rendered to the DOM. In the expense app, we compare the previous set of categories, expenses, and links to the next set, and set each component’s update attribute to true or false. The calculation is straightforward since the graph isn’t nested, but would most likely require a different approach if the data structure is more complex.
AppCalculationUtils.calculateUpdate = (prev, next) => {
_.each(next.expenses, (expense) => {
var prevExpense = _.find(prev.expenses, (prevExpense) => prevExpense.id === expense.id);
if (prevExpense) {
delete prevExpense.update;
expense.update = !_.isEqual(prevExpense, expense);
}
});
// do the same with categories and links
}(From utils/AppCalculationUtils.js)
We call the calculateUpdate function from Graph.jsx while calculating the graph data, and use the update attributes in the shouldComponentUpdate function of each child component. And if a component has update attribute true, call the D3 update code and return false to prevent React from calculating the diff again:
class ExpenseComponent extends React.Component {
shouldComponentUpdate(nextProps) {
if (nextProps.data.update) {
this.d3Node.datum(nextProps.data)
.call(CategoryVisualization.update);
return false;
}
return true;
}
}(From components/Category.jsx)
In Conclusion
React and Flux have really solved a lot of the problems that originally got us here, but they’ve opened up a whole new world of things to explore and improve. It’s gotten us to a good point where we can now seriously explore scaling — what if there are thousands and tens of thousands of elements that can be added and edited and deleted? Miles and I had a great conversation a few months ago about how the approach I outlined wouldn’t be performant at large numbers, so it has to be flexible enough to handle large numbers some other way. Aggregating? Filtering? It’s been amazingly fun.
There are small kinks to work out too: whenever the store fires multiple change events in quick succession, the graph recalculates, making the transitions twitchy. What’s the best way to throttle that? Or, how do we use Immutable.js for shouldComponentUpdate of nested components, so that we can know if a component’s children are entered or exited and need to finish the React lifecycle, or if only its attributes are updated and thus can terminate after calling the corresponding D3 code?
It’s been so much fun working on this blog post. All the research and thinking (and getting derailed by a “simple example app” that turned out to be quite involved) turned out to be great learning experiences, and I’m excitedly hoping that the post will foster interesting discussions I can learn even more from.
