Migrating to Version 4 of D3, part 1

At a recent Bay Area D3 User Group meetup, I gave a short talk on migrating to the new version of D3. I highlighted some of the changes in the version of D3 and shared a simple case study, with the hopes of helping others get started out.

This article is a slightly modified and expanded version of my talk.


There are so many changes in the new version of D3 that it’s just not possible to cover them all in a short discussion. So I’ll cover a few things that stood out to me as particularly useful.

The new version is full of big changes, and it’s not backward compatible. For a full list of changes, go to the source! There is extensive documentation on the changes in D3.

You could group the changes into three main types:

  • Changes to names → “the great namespace flattening”
  • Modular Library + NPM
  • Lots of Minor and Major restructuring, and additions

“The great namespace flattening”

Renaming (flattening) methods → Bunch of little changes, that you can’t ignore.

d3.random.normal --> d3.randomNormal
d3.random.logNormal --> d3.randomLogNormal
d3.scale.linear --> d3.scaleLinear

Modular Library + NPM

Big change. Can ignore this… for now.

Jump into it, if you’re already using npm or just want to dig in deeper.

  • D3 Reorganized into smaller modules → smaller downloads, customization
  • Can use npm and rollup.js to organize and pack-up your code as ‘plugins’.
  • Everything in d3 version 4 is a ‘plugin’, with imports and exports, so you can organize your code into reusable modules
We can still use monolithic version → may be easier for early experimenting!

New Goodies

d3.scan()

  • Linear scan of an array
  • Similar to d3.min() and d3.max(), which ignore NULL values
  • Returns index, instead of value of extreme

Very handy to have the index of an extreme! This opens up a lot of possibilities, that were more tricky before.

d3.set()

  • Extracts unique values from ‘column’ of array of objects
  • Not really new, but a little different

Still studying this one…It was a shim before.

var items = [ 
{total: 22, month: ‘May’},
{total:76, month: ‘June’},
{total: 28, month: ‘May’},
{total: 38, month: ‘May’}];
Var months = d3.set(items, function(d) { return d.month; }); // May, June

I often do this with python, while pre-processing…nice to have in D3!

data.columns

When parsing a csv or tsv file, you can now get

  • column names, in their input order
d3.csv(‘items.csv”, function(data){
 console.log(data.columns); // [“total”, “month”]
});

I often do this is python, while pre-processing….also nice to have in D3!

d3.stratify()

Tabular data → hierarchical

var root = d3.stratify()
.id(function(d) { return d.name; })
.parentId(function(d) { return d.parent; })
(nodes);

Can then pass this root to d3.tree and tree diagrams.

I just recently did this with python, and want try the new D3 way! Haven’t done it yet, but it’s on my list of things to learn more about.

timer.stop()

  • Can now stop a timer externally!!!

I use timers a lot, so I’m excited about this one!

Most motion in d3 done with transition.duration, but long running processes use Timer

Resources are freed more quickly than before. If you use Timer, I highly recommend reading up more in the documentation of the new version.

“Some usage patterns in D3 3.x could cause the browser to hang when a background page returned to the foreground. For example, the following code schedules a transition every second:
setInterval(function() {
d3.selectAll(“div”).transition().call(someAnimation); // BAD
}, 1000);
If such code runs in the background for hours, thousands of queued transitions will try to run simultaneously when the page is foregrounded. D3 4.0 avoids this hang by freezing time in the background: when a page is in the background, time does not advance, and so no queue of timers accumulates to run when the page returns to the foreground.”
--from the D3 CHANGES documentation

What does this mean? Why should we care?

Essentially, time now ‘freezes’ when in background. There are changes to queue management, and all sorts of fascinating goodies.

drawing to Canvas & SVG

  • Easier to draw the same path or shape to both svg and canvas.
  • Shapes were limited to SVG, but now shape generators have optional context for canvas

This is fantastic! Especially with systems combining svg and canvas…optimization opportunities abound!

“Under the hood, shapes use d3-path to serialize canvas path methods to SVG path data when the context is null; thus, shapes are optimized for rendering to canvas.”
--from the CHANGES documentation

Easing transitions

This is simple to update in existing code.

In version 3, we’d write

transition.duration(200).ease(‘sin-in-out’)

In version 4, it becomes

transition.duration(200).ease(d3.easeSinInOut)

This example, from the documentation, is a nice structure if you’re applying transition to several different groups at once!

var t = d3.transition()
.duration(750)
.ease(d3.easeLinear);
d3.selectAll(“.apple”).transition(t)
.style(“fill”, “red”);
d3.selectAll(“.orange”).transition(t)
.style(“fill”, “orange”);
--example from https://github.com/d3/d3-transition

Now that we’ve covered some of the new goodies in version 4 of D3.js, let’s switch gears and step through migrating an existing visualization to version 4.

In this next portion of the talk, I shared my experience with migrating one of my own projects.


Case Study: Scatter Plot

While I start figuring out how to modify existing D3 projects to version 4, I want to keep things simple. There’s plenty of time for the big changes, once I figure out how to get started and migrate simple projects.

Recently, at work, I’d been given a data set and asked to see what I could ‘come up with’. This scatter plot is one of the pieces I developed.

The data set was multivariate, so I had a lot to play with. I didn’t know much about the data, so I started looking for patterns visually.

My own explorations involved changing which variables to chart. I organized my code so that it was easy to just swap out which variable would be drawn.

Initially, I changed the variables directly in the code. Once I built simple controls, it was easier to switch between views and talk about patterns with colleagues.

The resulting visualization is a relatively simple scatter plot with a legend, and an interface for triggering transitions between chart variables. It’s a good case study for converting an existing visualization to version 4 of D3, because it is fairly simple but has some slightly complicated features.

Migration

1. Get new library

Seems obvious, but we gotta start somewhere.

2. Replace script source tag in project code

<script src=”d3.min.js”></script>

becomes

<script src=”d3.v4.min.js”></script>

I could keep it as ‘d3.min.js’, and just save the version 4 library as d3.min.js. However, there are some advantages for specifying v4 in your code. First of all, specifying v4 helps avoid confusion during migration. We can easily verify which version of D3 we’re working with.

Another advantage to specifying v4 in your code is that blockbuilder uses the tag to categorize examples. If you are sharing your bl.ocks with version 4 of D3, you can make it easier for others to find them.

3. Check to see if anything works

Does anything work, if I just change the library? The documentation says that version 4 is not backwards-compatible. But will some things work, since this is such a simple project?

Nope! Nothing works, except for html elements ‘hard-coded’ in the document.

OK, let’s get to work updating to V4!

4. Start with namespace changes…’the great namespace flattening’

Reading through the changes documentation, it seems like the simplest change is the namespace flattening. So that’s where I start.

Quickly scanning my code, I notice scales. Fixing is a simple ‘find and replace’.

d3.scale.linear

becomes

d3.scaleLinear

Nothing is rendering yet, so I move on to migrating the code for rendering axes.

5. Axes:

Slightly more complicated fix than flattening namespaces, the fix is not just a find/replace of syntax. Axes have default styles now.

svg.append(“g”)
.attr(“transform”, “translate(0,” + height + “)”)
.call(xAxis) // remove from earlier code, now style default

becomes

svg.append(“g”)
.attr(“transform”, “translate(0,” + height + “)”)
.call(d3.axisBottom(x))

Very nice! Simplifies things moving forward. It does make me wonder about customizing styles beyond the defaults, but I’m going to save that for another time.

I’m not done with Axes yet, though. Because this project changes the axes, when switching between variables to chart, I have to rewrite the function that handles this. In my code, UpdateChart() updates positions of points, changes axes, and updated axes labels.

So I applied the new axis syntax, with default styles, and tested. The transition-ease raised an error , so I commented it out. I’ll come back to it later.

  • Changed call to new axis syntax.
var axisX = svg.select(“#x_axis”)
.transition().duration(200) //.ease(“sin-in-out”)
.attr(“transform”, “translate(0,” + height + “)”)
.call(d3.axisBottom(x)) //.call(xAxis)
  • Used id instead of class to select label, and set new value from button
var axisX = svg.select(“#x_axis”)
.transition().duration(200)//.ease(“sin-in-out”)
.attr(“transform”, “translate(0,” + height + “)”)
.call(d3.axisBottom(x))
d3.select(‘#xlabel’).text(xValue) 

With the new axes, labels are now general elements, and not attached to an axis. So I’m selecting the label by id, and updating the text value.

6. Easing transitions:

Now that the axes and labels are working properly, I can address easing transitions.

Transition easing is a little different in v4, but a fairly simple fix.

svg.selectAll(“.dot”)
.transition().duration(200).ease(“sin-in-out”)
.attr(“cx”, function(d) { return x(d[xValue]); })
.attr(“cy”, function(d) { return y(d[yValue]); })

becomes

svg.selectAll(“.dot”)
.transition().duration(200).ease(d3.easeSinInOut)
.attr(“cx”, function(d) { return x(d[xValue]); })
.attr(“cy”, function(d) { return y(d[yValue]); })

With these changes, my visualization functions the way it did before the migration. The source code for this migrated chart is on github, with its imperfections, warts, and all.

7. Now what?

While I’m going to pause here, there are still a lot of things I could do to make this migration more thorough. I could even take advantage of the modular nature of this version of D3. I could modify it and make it into it’s own plugin, or break it into smaller reusable modules and use rollup.


Finishing up with some thoughts on what makes good code

At a bare minimum, good code is readable. You can come back to it, understand it, and modify it.

D3 is moving more in this direction. Check out Mike Bostock’s article. It’s a good read! medium.com/@mbostock/what-makes-software-good

With all this improved readability, and other improvements, D3 is becoming an even better suited tool for more of my projects! More flexible and extendable.


We can write code to be readable, and then use tools for making it more compact.

Tools

Uglify: JavaScript compressor/minifier written in JavaScript

  • To minimize code, and remove comments (or preserve them).
  • Take a more pre-compiled approach. Write code for human eyes, and compile to something the machine/browser reads.

Rollup:

  • To combine multiple javascripts into one file.
  • Can then organize your own codebase in modules, combine, and extend them.

I’m excited about this…was working on python project to do this for me…don’t have to now

Resources:

Mike Bostock’s bl.ocks

https://bl.ocks.org/mbostock

Lots of examples from the creator of D3! Most recent ones are V4s

Blockbuilder

Can search for bl.ocks by version of D3, author, etc: http://blockbuilder.org/search#d3version=v4

Check out blockbuilder…amazing resource!

Thank you!

Adding new resources, as I find them:

D3 V4 — What’s new? by Irene Ross, of Bocoup