Part 1: How to analyse and modify your programs
Metaprogramming is a powerful but under-appreciated tool. Even though many widely used development tools use it under the hood, interest in metaprogramming is steadily declining (at least according to Google Trends data):
There could be many reasons behind the trend. It used to be a fairly popular tool in the fields that nowadays Machine Learning is thriving in. As development tooling gets more and more sophisticated, the need to investigate such topics decreases. Whatever the reason, I strongly believe that this forgotten art is still relevant and can bring a lot of benefits once mastered.
What is metaprogramming?
Wikipedia defines metaprogramming as a “programming technique in which computer programs have the ability to treat other programs as their data. It means that a program can be designed to read, generate, analyze or transform other programs, and even modify itself while running”.
In other words we can use metaprogramming to inspect, modify, or even generate code from scratch. We can also use it to write self-modifying programs — a use-case outside of the scope of jscodeshift, but one I covered briefly in a previous article:
Code which modifies original source code is commonly called a codemod. Plenty of libraries have their codemods publicly available on GitHub — this is usually a good place to start and learn how to write your own.
The following diagram shows the general overview of how jscodeshift works in practice. Firstly, the source code we wish to modify is transformed to an AST, then the codemod is applied to add/remove/modify the nodes. As a result, another tree is created. We can then convert the tree back to code form and save it.
What is an abstract syntax tree (AST)?
Let’s start with a simple example:
and see an AST representation of it:
We can see that our code has been transformed into simple tree with a
BinaryExpression (which represents addition) as a node and two arguments (left-hand and right-hand) as the child nodes.
Now let’s consider some more complex code and its AST representation:
In this example the tree grew quite significantly. As you can see our conditional expression has been transformed into the node containing 3 children nodes:
- test: the condition under which the “consequent” expression will be executed
- consequent: code that gets executed if the test subtree evaluates to true
- alternate: code that gets executed otherwise
We can explore the tree further and inspect each of these paths separately. Our
x > 3 statement has been converted to
BinaryExpression subtree with two arguments — similar to our
console.log(x) expression got expanded into
MemberExpression which is an AST representation of taking a property from the object (
console.log) and then combining it with the arguments (
x) under the
If you want to inspect your code’s AST, you can use AST Explorer to generate an AST representation. You can explore the tree in the JSON format on the right. When you hover over a node the corresponding code will get highlighted.
This form of code is really easy for machines to analyze and modify. In fact it’s exactly what jscodeshift uses to modify!
Real life examples
Before we start writing our own code modifications, we should take a look at what codemods can do and where they are useful. Lots of them are documented really well.
Migrating to Jest
The Jest migration guide recommends using jscodeshift to automate the migration process from popular test frameworks like AVA, Chai, Expect.js, Jasmine or Mocha to Jest. The codemods themselves can be found on the jest-codemods GitHub. There are many great stories describing how this codemods helped developers to migrate their projects faster, such as when Kent C. Dodds migrated a codebase from AVA to Jest at PayPal.
React breaking changes
React includes codemods as part of the migration guides between versions that introduce breaking changes. These guides (in theory) allow you to migrate between versions with just a single command. In practice, some manual fixing is usually needed but having a codemod speeds up the process significantly.
Gatsby migration v1 -> v2, aka codemods are hard
On the other hand, codemods are not always that helpful. Recently I tried to migrate an old Gatsby v1 project to the latest version using the official codemods. Unfortunately, even though our codebase was fairly simple, some codemods didn’t work and others generated invalid code. Writing a good codemod is not straightforward given that it needs to run on all possible codebases. In our case migrating
Link components broke the app because we had an abstraction on top of Gatsby’s class. After the migration all of the imports were duplicated and we had to fix them manually.
Migrating lodash / underscore to native functions
Other useful codemods and resources
There are many other codemod resources, notably:
- React codemods — a repository for migrating React projects
- “Awesome” jscodeshift — a repository with plenty of articles, talks, tools and codemods
This is article is just an introduction to codemods. Stay tuned for the second part in which we will create three codemods that format code correctly and search for data leaks.