Running stuff when needed
I want to introduce run-when, a CLI to run tasks based on Git diffs changes.
Scenario 🏜
Recently I have been working quite a lot in a mono-repo environment. I won’t explain here what is cool or not about it, instead, I want to go through a specific problem we faced and how we came up with a solution.
Time is money, well, time is life. When you are working in a team you want to build stuff fast and with quality. You get quality by writing tests and running them into a continuous integration environment, this ensures you don’t break existing things when you add new functionality. But in the other hand, this takes time, having to run a test-suite on every single change for every single package makes things slow.
Let’s say you have 5 packages in the mono-repo, each of them takes about 5min to pass a successful build (fetching dependencies, building app, running specs, etc). This means 5x5min=25min, not the best time, right? Now let’s say, one of those packages have integration tests, which sometimes end up taking >10min. Other package has a flaky test, which fails sometimes, things start to get even worse…
Solution 💡
Run it, when you need it
Why do you need to run tests for a package which you haven’t change? How about just running specs based on Git? That was the motivation of run-when, do a git diff
and check if anything within your package has changes, then, and, run stuff:
$ run-when '["packages/component-a/**"]' 'cd packages/component-a && yarn test'
- First parameter is an array of Blobs to match files against.
- Second parameter is the command to run.
Programatic way:
As seen above, the library has full blob support (thanks to multimatch), asynchronous tasks, and optionally you can pass changedFiles, just in case you don’t want to use the default command:
git diff --name-only origin/master
Which by default returns a list of changed files in your branch against master.
Testing the tool 🔬
It can get tricky to test CLI tools if you haven’t thought about that when you designed the tool. In the past, I had built other tools but never really spent time thinking about how to test them properly… no this time!
What I did different this time, was to first build the tool as if it was going to be used as any other package, like import runWhen from 'run-when'
. That way, I could focus on how I wanted other people to use the tool and what public api to expose. Then, the cli part becomes just a thing wrapper around that.
- Unit tests: Somehow I needed to fake which files have been modified, to be able to have a good coverage. To do so, we extended the existing api in a way that the user could specify an array of modified files. That way we made the code “more testable”:
- CLI/Integrations test: For this one I actually wanted to test the whole thing. To do so, I simulated some changes in an existing file and tested some globs against it. Have a look here.
Attention to the details 💅
One of the main use cases for run-when is running a testsuite when some source files have changed. This may take time… ⏰
This was the first output we were giving to the user:
Not good right? you probably want to see the progress of those specs in your CI while they are running. We achieved that piping the stdout stream into the console:
Finally, it is sad if you do all this work but don’t keep the original colors of the task. Passing FORCE_COLOR to the env, will do the trick:
Other fancy stuff 💘
One thing I love about doing open source is that you always learn new things. It gives you a new opportunity to play with something you probably can’t in the daily bases. Here is some things I learnt this time:
- Node & NVM: I used Node 8 (currently LTS 🙌) for building this library, it may look a bit bleeding edge, but it all depends on your use case. Since this is a dev tool and is likely going to be run on a CI it was safe for us to target that version. You can easily enforce that by using a .nvmrc file in your project and letting
$ nvm use
do the rest. - Promisify: Node 8 has a new utility function:
util.promisify()
. It converts a callback-based function to a Promise-based one. So there is no need anymore to pull external dependencies or build custom helpers to have a nice promise oriented way of doing things:
- async/await: Code becomes more readable using await all over the place, specially when used in a ternary operator or with Jest!
- Flow: While I enjoy using Typescript, and is the type checker I use at work everyday, this time I wanted to play a bit Flow. It has excellent type inference and painless setup, since its just a babel plugin, you can just add it to your existing project and will work out of the box with Jest as well.
- Jest: I was already using Jest before, but only for browser env (React + Enzyme). This time was all about Node, and I can’t fall more in love with it.
- Destructuring: This is one of my favorite js features. This was not the first time I use it, but it was the time I had more fun time with it! In the following example, Im destructuring an array into const using the index and defaulting to some value 😵
- Travis install: I discovered that by default, Travis does a shallow clone, this means that is not fetching the whole GIT history.
git clone — depth=50 — branch=master https://github.com/zzarcon/run-when.git
This is fine 99% of the times, as you don’t care about that info, but in this case you actually need an up-to-date master to compare your changes with. You can achieve that doing:
Conclusion
I hope you all enjoyed reading the article just as much as I did playing with the tool! If you have any question or improvement please feel free to reach!