Automate migrations with Nx schematics
In a world where client technologies keep changing and moving forward, we want an easy way to maintain and keep our projects up-to-date. That is exactly what custom CLI gives us: a convenient way to do repetitive tasks like migrations or adding new parts to our project that can be automated.
There are plenty of usages to custom CLI:
- Generate new components or services
- Migrate and update code to fix breaking changes or move to another implementation
- Adding new framework or configuration to a project.
In this post we will focus on migrations tools and will go through an example of building a simple migration tool using Nx schematics. The real advantage of automating the migration process is that it save hours of work and reduces bugs that we usually find after manual migrations. It allows us to make these updates more often in our project since it’s safer and we don’t have to spend as much time on it.
The first migration we wrote in Bob was an update to our storybook. We used an old version and wanted to update it but it had breaking changes.We needed a way to refactor all our existing stories (more than 150) to fit the new version. The issue was that the new version was more “nice to have” than a “must” so we didn’t want to spend a lot of time on it and decided to try and do it with custom CLI. The result was amazing, a tool that took a day to write, saved us at least a few days, and made this task possible.
The tool we will build
We are going to show an example of a migration tool used to wrap a third-party lib we use in our code. The example is based on a real use case we had in Bob and will show how we used migration to solve it.
In our client code we are using the Lodash utilities lib. We wanted to remove some of the usages and replace them with our implementation. We also found many cases where Lodash was abused inside the project. Simple functions like map or forEach that exist on array were used in Lodash instead.
To solve it, we wrote a tool that scanned our project, added all Lodash methods into one place, and replaced the usages all over the project to be from that file. This tool will be the example of this post, to understand and learn the concepts of automating migrations in our project.
What is Nx schematics and why should we use it?
To start writing custom schematics, we need a place in our project to add our tools and a way to run them. Angular schematics gives us the option to quick start, it’s easy to configure, and many guides can be found online. However, for the long term, I recommend using Nx schematics to wrap your custom CLI.
It extends angular schematics with more utils, and unlike angular, you don’t need to build the command to have it updated.Nx takes care of it for you, so you just need to write the code.It will automatically update so you are always running the latest version of the command.
To work with Nx schematics, you need to have Nx configurable inside your project.
You can see how to generate Nx workspace or migrate an existing one here.
Since we use Nx in Bob, I will demonstrate how to create the tool with it. However, the logic and schema can be run in angular schematics, as well, so if you don’t use Nx in your project, you can skip to the next step.
*If you are not using Nx and skipped the previous step, you need to add a new folder named ‘lodash-migrator’ inside your tools folder and add two new files inside it, schema.json and index.ts (see screenshot attachment above).
In this example, we will use angular schematics, and not Nx,. This will allow you to run the command in angular schematics without Nx, but the explanation will assume Nx is being used.
Creating the tool
Step 1 — Create the tool
Nx provides us with a very easy way to create tools, we just need to give the workspace schematic generator our new tool name:
The tool creates two files under the generators folder:
- schema.json — describe the command options
- index.ts — the command logic
Step 2 — Edit the schema.json
The schema.json describes how our command will behave and look inside the terminal. We can control the parameters types, default values, and much more with this file.
In this example, we will use angular schematics to run the command and Nx just as a wrapper to take care of the command build. We will remove the “cli’’ key so we won’t use NX schematics. In addition, we don’t need any parameters in our tool, so we will remove the “name” key from “properties” and “required”.
Auto generated schema:
Updated schema after the changes:
Step 3 — Edit the index.ts
We will edit the generated index.ts file to work with angular schematic instead of Nx:
Nx schematics generated file:
Replace it with:
We are exporting a default function that will have the command logic inside it. Right now, it only returns an empty chain and basically does nothing.
Chain gives us the option to run multiple updates to our tree and gives us an easy way to separate our tool logic.
Next we want to add the logic that will update the tree. To do this, we need to create a function that returns a Rule, just like the default function, and use it inside the chain:
Now that we have all the template we need, let’s add the core logic of our command. We want to replace all the usages of Lodash with a new wrapper. The steps of the command should be:
Scan all .ts files and check if Lodash is used. Then, get Lodash imported methods and replace the import to the new file.
Create the wrapper file based on any Lodash methods you found.
We need a way to scan all our project files. tThis can be done using the tree object from angular schematics:
Let’s use it to get all our files and check if they include Lodash import.
We will create a set to save all the used methods and will add them inside the file scan.
For each file with Lodash, we want to update its import and get all methods. In case Lodash was imported more than once, we will replace all the occurrences using regex. Afterwards, we want to scan all Lodash imports inside the file and add them to our set.
The logic above splits all the imports. For each import, it takes all the content inside the {} and extracts the used functions from there.
After we handle all the files, we still need to create the wrapper. This will be done by looping all Lodash methods that we found. It will also generate the next code for each one of them:
Creating the wrapper logic in our tool:
Full code of the index.ts:
Step 4 — Run the command
Once we finish the command, we can run it using Nx.
npm run workspace-schematic — lodash-migrator
After running the command, it will do two things: the first is to update all our file imports; the second is to create our wrapper.
We can see the updated files inside the terminal.
And the generated wrapper:
*Edge cases:
As you probably noticed, this migration only handles ‘lodash’ imports and ignore imports like ‘lodash/…’. In Bob, we had 750 files that import ‘lodash’ and only 5 files that used inner imports from the lib. Handling these cases manually made the tool easier to write.Since it was only five files to handle, it would take more time to try and build the logic to automate all the cases.
Conclusion
Once you set up the schematics, especially when using Nx, it’s very easy to set up tools like the one above to automate and improve your dev experience in many ways.
Don’t wait for it to happen, make yours and your team’s life easier by bringing the magic of custom CLI into it.
