The Trouble with D3

Ian Johnson
Jun 12, 2018 · 7 min read

Recently there were a couple of threads on Twitter discussing the difficulties associated with learning d3.js. I’ve also seen this come up in many similar conversations I’ve had at meetups, conferences, workshops, mailing list threads and slack chats. While I agree that many of the difficulties are real, the threads highlight a common misconception that needs to be cleared up if we want to help people getting into data visualization.

The original thread that spurred quite a bit of discussion and some salient points.
A thread with lots of excellent hard-won perspective.

The misconception at the heart of these threads is that d3 and data visualization are the same thing. That to do data visualization one must learn and use all of d3, and to use d3 one must learn all of data visualization. Instead, I like to emphasize that d3 is a toolkit for communicating complex data-driven concepts on the web.

What I want to get across here is how we can get a more holistic view of d3’s role in web-based data visualization. Let’s use a metaphor inspired by Miles McCrocklin where data visualization is likened to building furniture. All kinds of people might get into building furniture, for all kinds of reasons, especially when they see the beautiful things other people are making:

The Eames chair is considered a masterfully designed chair
There are many aspirational data visualizations made with d3

People see the impressive output and naturally desire the ability to make it themselves, they ask how it is done and often hear “it was made with d3.” This is the start of the problem, because when someone hears that it was made with d3, they think “oh, I should go learn d3”. They go over to the documentation and see something like this:

d3’s API

Many of these tools seem baffling, they require knowledge about woodworking and processes we’ve never thought about before, or even knew we might need to think about. We feel overwhelmed and discouraged, it seems the path to something that seemed within reach is long and treacherous.

This is where I believe we can change things for the better, rather than changing the toolset we can guide people based on their goals along more suitable paths for them. Let’s examine a few common situations where people find themselves wanting to do interactive data visualization and how we might plot a better course for each.

The designer

What part of this is JavaScript? What part is specific to d3? What is an asynchronous request? What is this DOM I keep hearing about?

For these folks, d3 offers great power and flexibility, but first they must learn some foundational technical skills to operate in this environment. I often recommend Scott Murray’s excellent d3 tutorial (and book) which covers basic HTML, CSS and JavaScript concepts. I also recommend experimenting with exporting SVG from design tools like Illustrator and Sketch and imbuing them with interactivity and data magic in the browser.

When starting out, I often encourage designers not to focus on the enter/update/exit pattern, reusability or performance concerns. It’s much more helpful to focus on getting the desired output, once you have something close there are lots of friendly folks that can help you make it more performant or robust.

The analyst

What is the difference between SVG and Canvas? What is the JavaScript equivalent to Pandas/Tidy? Why can’t I draw a line chart with an SVG line? What is this “d” attribute on a path?

For these folks I also recommend a primer in web development to familiarize themselves with concepts like the DOM. Again, my favorite starting point is Scott Murray’s d3 tutorial (and book). I would also recommend a crash course in JavaScript and JSON, exporting data from their normal environment as JSON for visualizing in the browser.

When starting out, I often encourage analysts to ignore a lot of d3’s utility functions, as they are probably more familiar with the powerful functions in their own environments. Instead, I think its best to focus on exporting the data into an easy-to-consume JSON or CSV format that matches existing examples.

The software engineer

What is this enter/update/exit business? Why are you messing with my DOM? Transitions… How do I unit test those?

Many developers will already be intimately familiar with the DOM and JavaScript, so my advice is to actually try and ignore the parts of d3 which focus on the DOM. Instead, become familiar with some key utilities for data visualizations like d3-scale. D3 is broken up into many smaller modules so it’s pretty easy to cherry-pick the functionality you want to use.

I also emphasize separating the layout of data from the visualization, so using a module like d3-hierarchy you can generate a data structure with d3 and then render it into the DOM using your framework of choice.

Silver bullets

I personally think of web standards as the lowest common denominator for global communication. The graphics APIs are not ideal but if you want to instantly distribute your data-driven experience to billions of people I think it’s reasonable to pay the price of a relatively steep learning curve. The underlying concepts of 2D graphics, visual design, user experience design, information architecture and programming all transfer directly to many other endeavors besides data visualization.

But sometimes, a chair is just a place to sit, we don’t have time or money to care that much and IKEA will do just fine! In those cases there are plenty of charting libraries that only need a little bit of configuration to get going.

Sometimes this is the only tool you need.

Elijah Meeks has made a great map of the d3 API that breaks down the toolbox into useful categories in his recent article.

from Elijah’s D3 is not a Data Visualization Library article.

I’ve also attempted to map out the d3 learning landscape in my article The Hitchhiker’s Guide to D3, which gives some links and starting points for what I believe are some of the more essential concepts.

The journey isn’t easy, but it can certainly be an adventure!

A while back I interviewed a handful of data visualizers who learned d3 in the process of expressing themselves and the datasets they cared about. The common theme was that they had started with goals. They learned what they needed from d3 along the way to achieving those goals.

So grab a map and plot your own course through the vast world of Data Visualization. You can find some trails others have blazed with Blockbuilder search, try out JavaScripts very own Notebook environment Observable, and join over 3,000 like-minded chair makers, I mean data visualizers on the d3 slack channel.

Good luck, I look forward to seeing your visualizations!

I’d like to thank Erik Hazzard, Kerry Rodden, Zan Armstrong, Yannick Assogba, Adam Pearce and Nadieh Bremer for their feedback on this article.


JavaScript news and opinion.