Effective JavaScript Codemods

Tool assisted code modifications can help evolve complex systems incrementally and aid in maintaining the health of large codebases.

Update: I gave a talk “Evolving Complex Systems Incrementally” about this topic at JSConfEU 2015.

We have tens of thousands of JavaScript modules at Facebook. When an engineer makes breaking API changes we transform all of our JavaScript to new code. To deprecate and remove legacy APIs Felix Kling and I developed a tool called jscodeshift for doing AST-to-AST transforms for JavaScript.

History

Early this year we embraced Higher-Order Components (HoC) when we were building a new API for Relay. We used to have a React mixin that added all the Relay-specific functionality but it used internal React APIs and bloated all components with Relay features. We wanted to move to a new container API that would wrap every React component with a Relay.createContainer call. Because we were using Relay in production in a few different products by that time we had to build strong transformation tools that could be applied to our codebases to make the changes automatically.

We could have built this using regexes but, given the number of changes we had to make, and the subtle code style variations in different modules and products, it seemed like a bad idea. I wouldn’t have been confident with my changes.

This is where jscodeshift comes in — instead of making string-to-string transforms it can transform a JavaScript AST (Abstract Syntax Tree) and then pretty-print the output and preserve code style.

me in a bow tie announcing the anniversary of JavaScript modules at Facebook. And then telling the story of how I removed them.

Toolbox

jscodeshift is a toolbox. It brings together babel’s parser (currently called babylon), Ben Newman’s ast-types for node creation and recast for pretty-printing — this is important because we care about printing code with certain formatting rules applied. It wraps these tools up nicely and provides a unified API to find, filter, map and replace AST nodes. It also comes with a runner/worker feature that can apply transforms to thousands of files in parallel. If you have never worked with JS ASTs you can use the JS AST Explorer built by Felix to explore AST nodes.

Consider a hypothetical module called merge. We used to have a module like this at Facebook. It got replaced by the object spread pattern that inlines properties of an object in an object expression, like this:

merge(a, {b: 1}, c); // Old
({...a, b: 1, ...c}); // New

Because merge could take any arguments and we had thousands of callers this was impossible to do with regexes. There might be functions defined as part of object expressions that themselves have merge calls, consider this:

merge(MyObject, {
  method: function(a, b) {
    return merge(a, b, {c: 1});
  }
});

But more importantly, the module in our codebase was called merge and even though our code conventions say that local aliases should mirror the module name we do not enforce this — the module merge might be called m, mergeAll or javascriptRules in some places where it was being used.

With jscodeshift building a transform for this is easy. Here is the code:

The above example omits searching for the module and removing the require statement. The full codemod is part of js-codemod, a collection of useful transforms for use with jscodeshift. We integrated jscodeshift with AST Explorer so you can play with the transform live. I’m also working on a Nuclide transform plugin with live editing.

The codemod uses find to search for all CallExpressions of merge and replaces them with an ObjectExpression that uses the new SpreadProperty syntax or it inlines properties from other ObjectExpressions directly. If the ast-types way of creating nodes using composition looks foreign to you, we also added the ability to use template literals to write JavaScript code that matches the expected output. Check out this test in jscodeshift for an example on how to convert for loops into while loops.

Running this codemod on a JS codebase is easy: just point it at a directory and it will transform all of the files in the directory hierarchy.

$ jscodeshift -t rm-merge.js html/js

That’s it. jscodeshift will do its thing and apply the transform file on every JS file it can find in the html/js folder. As a bonus, transform files are automatically transpiled using babel, for easier development. Afterwards we can inspect the pretty-printed code, commit it and the merge module will be gone from the codebase for good!

Disrupt

Sometimes large changes are really hard to test and they can affect thousands of files and tens of thousands of lines of code. There is no point in trying to make changes if it disrupts every engineer in your company. The idea is to not break things and to build transformation tools that we can be confident in. In the past few weeks I’ve seen engineers at Facebook write codemods for the first time and it was great to see how confident (and happy!) they were when they applied them to all of our JavaScript.

me after learning a new superpower, like how to transform thousands of files confidently.

Favorite Editor? JavaScript!

We built react-codemod which includes a React.createClass to ES2015 class transform. This transform can be applied to large codebases automatically to modernize code. This has influenced our open source philosophy at Facebook — we are running codemods like these internally to transform our code for the future. For widely used frameworks such as React we are trying to provide codemod tools to ease the burden on developers when we make changes to the core APIs.

We think that tool-assisted code modification is set to profoundly transform — no pun intended — the way people evolve and maintain very large scale codebases. We ran dozens of codemods all over Facebook’s JavaScript codebases and we are very excited to share this toolbox with the open source community and improve them together.