D3 is Now Modular
“D3 is now modular”. That’s how the D3.js v4.0.0 release note starts. What does it mean? Why should I care? It’s important as a community that we discuss it so we can make the best out of it. So at the D3 Unconf 2016, one session was about this concept of modularity. Here’s my summary.
D3 v4 is a collection of modules you can use independently, with minimal dependency between them, all neatly isolated in their own repo. It’s a good illustration of one definition of modularity: having code as a collection of minimal functional units, tightly encapsulated and exposing a clearly defined API.
One advantage of this new modular approach is to encourage developing and owning more granular modules. The community was already providing a lot of charts libraries, some d3 layouts and plugins, some code structures like d3.chart and Koto, some higher-level abstractions like D4, but not much core or lower-level features. The closest we have from core feature extensions are some pretty cool monkey-patch and extensions like d3-jetpack and D3 Kit. But now, contributing to core features or any other abstraction levels just got easier.
Bundle
When a javascript developer talks about modularity, he often refers to the mechanism of wrapping and loading modules, like using AMD, CommonJS, ES6 imports, etc. Indeed, one characteristic of modules is how they can be bundled together. D3 modules can be loaded independently from each Github repos or from npmjs. Or the official bundle can be loaded at once. But you can also build your own version. Why would you want to do that? Maybe you want to add some modules that are not in the official package, like the useful d3-selection-multi. Maybe you want a smaller package only containing what you actually use. A good example of a custom package would be to bundle all modules that are independent from the DOM, to use on servers without using a dom implementation, on Web Workers or even with React without interfering with its virtual DOM lifecycle. D3 v4 uses Rollup to bundle using ES6 modules syntax. Here’s a simple example of how to do it and a more complex one. I especially like this tutorial about how to make a plugin, and of course the official one from Mike Bostock.
I’m not a big fan of scaffolding: compilers/transpilers, code loaders/bundlers, etc. I often use a simple and well documented module pattern that I simply concatenate/minify on deployment for production use. But whatever technology you use, IMO the power of modularity is not about loading and bundling, but about encapsulating in a way that small modules can form bigger modules on another level of functional abstraction. For example, D3 is a “visualization kernel”, a very thin layer on top of the DOM API and other web technologies. Then the community is building tons of examples with it. But between low-level D3 modules documentation and standalone examples, there’s a whole range of modularization that is mostly missing. Let’s look at these levels of encapsulation that are harder to find example of: components, charts, libraries, views, apps.
Components
A common pattern to encapsulate charts is the Reusable Pattern, which is nothing more than a “closures with getter-setter methods”. NVD3 is one of the most popular charts library using it. But IMO the best example of this reusable pattern is the d3-axis. I described this pattern, how to write unit tests for it and how to use it in a real-world app in a short book we wrote a while ago: Developing a D3.js Edge. Interesting modifications to this pattern where proposed . I recently wrote a minimalistic example of a simpler pattern I use for encapsulating charts and widgets, in that case a simple todo app inspired by TodoMVC. But other patterns can work well too, like traditional object-oriented code used by one of the oldest D3 charts library around called Rickshaw.
Many patterns can be used, but what I want to emphasize is how encapsulation can happen at the “component” level. I define a component as any reusable building block to build charts. An axis is a good place to start, as it sometimes is the most complex part of the chart. For example, when I designed Micropolar for Plotly, I realized that most of my code was describing a very configurable polar axis, the rest being simply graphical marks and helpers around it. Plottable.js is working at this type of component level. D4 proposes an interesting way to mix in and out components and features. Interstingly, some of these patterns look a lot like Protovis, the project from which D3 derived.
During my PhD, I studied ways to think all visualizations as assemblies of components. Someday I should make something out of all the material I have about the design space of hybrid visualizations. Here’s one paper we published at Eurovis about chart components.
When working on Firespray for Boundary, and then on Cirrus.js for Planet OS, I experimented with a nice pipeline pattern that I still use today to encapsulate low-level components and pipe them together to describe charts. I used it in multiple contexts, I even tried a version using Rollup. I will describe how I solve various challenges using this pipeline pattern in a follow-up blog post.
Charts, Libraries, Views, Apps
Various patterns can be used to describe how the components will be assemble to form a chart, from factories to mixins to inheritance. Sometimes the assembly involves a more general framework, like React, Backbone or Angular. I prefer to keep my charts code in plain javascript and D3, then wrap it with React for example. But that’s personal preference. What is important here is to emphasize that there are multiple levels of encapsulation to take into accounts: toolkits, components, charts, chart libraries, UIs (e.g.: dashboards), applications as a collection of UIs, software ecosystem, etc. At each level, there are choices to make that depends on a lot of different factors. Data piping, events passing, error handling, state managing, are all bringing different constraints to the architectural choices. But at every level, the only feature I always keep in mind is modularity, encapsulating functional blocks that can be assembled to form larger functional blocks, until I reach the final functional block: the actual end-user product.
In conclusion, I’m glad the community is sharing more patterns, code architecture and modules at various levels, adding diversity to the already awesome collection of examples, chart libraries and plugins ecosystem.
