Running stuff when needed

Me waiting for CI to finish

I want to introduce run-when, a CLI to run tasks based on Git diffs changes.

Scenario 🏜

Recently I have been working quite a lot in a mono-repo environment. I won’t explain here what is cool or not about it, instead, I want to go through a specific problem we faced and how we came up with a solution.

Time is money, well, time is life. When you are working in a team you want to build stuff fast and with quality. You get quality by writing tests and running them into a continuous integration environment, this ensures you don’t break existing things when you add new functionality. But in the other hand, this takes time, having to run a test-suite on every single change for every single package makes things slow.

Let’s say you have 5 packages in the mono-repo, each of them takes about 5min to pass a successful build (fetching dependencies, building app, running specs, etc). This means 5x5min=25min, not the best time, right? Now let’s say, one of those packages have integration tests, which sometimes end up taking >10min. Other package has a flaky test, which fails sometimes, things start to get even worse…

Solution 💡

Run it, when you need it

Why do you need to run tests for a package which you haven’t change? How about just running specs based on Git? That was the motivation of run-when, do a git diff and check if anything within your package has changes, then, and, run stuff:

$ run-when '["packages/component-a/**"]' 'cd packages/component-a && yarn test'

• First parameter is an array of Blobs to match files against.
• Second parameter is the command to run.

Programatic way:

As seen above, the library has full blob support (thanks to multimatch), asynchronous tasks, and optionally you can pass changedFiles, just in case you don’t want to use the default command:

git diff --name-only origin/master

Which by default returns a list of changed files in your branch against master.

Testing the tool 🔬

It can get tricky to test CLI tools if you haven’t thought about that when you designed the tool. In the past, I had built other tools but never really spent time thinking about how to test them properly… no this time!

What I did different this time, was to first build the tool as if it was going to be used as any other package, like import runWhen from 'run-when' . That way, I could focus on how I wanted other people to use the tool and what public api to expose. Then, the cli part becomes just a thing wrapper around that.

• Unit tests: Somehow I needed to fake which files have been modified, to be able to have a good coverage. To do so, we extended the existing api in a way that the user could specify an array of modified files. That way we made the code “more testable”:

• CLI/Integrations test: For this one I actually wanted to test the whole thing. To do so, I simulated some changes in an existing file and tested some globs against it. Have a look here.

Attention to the details 💅

One of the main use cases for run-when is running a testsuite when some source files have changed. This may take time… ⏰

This was the first output we were giving to the user:

Not good right? you probably want to see the progress of those specs in your CI while they are running. We achieved that piping the stdout stream into the console:

Finally, it is sad if you do all this work but don’t keep the original colors of the task. Passing FORCE_COLOR to the env, will do the trick:

Other fancy stuff 💘

One thing I love about doing open source is that you always learn new things. It gives you a new opportunity to play with something you probably can’t in the daily bases. Here is some things I learnt this time:

• Node & NVM: I used Node 8 (currently LTS 🙌) for building this library, it may look a bit bleeding edge, but it all depends on your use case. Since this is a dev tool and is likely going to be run on a CI it was safe for us to target that version. You can easily enforce that by using a .nvmrc file in your project and letting $ nvm use do the rest.
• Promisify: Node 8 has a new utility function: util.promisify(). It converts a callback-based function to a Promise-based one. So there is no need anymore to pull external dependencies or build custom helpers to have a nice promise oriented way of doing things:

• async/await: Code becomes more readable using await all over the place, specially when used in a ternary operator or with Jest!

• Flow: While I enjoy using Typescript, and is the type checker I use at work everyday, this time I wanted to play a bit Flow. It has excellent type inference and painless setup, since its just a babel plugin, you can just add it to your existing project and will work out of the box with Jest as well.
• Jest: I was already using Jest before, but only for browser env (React + Enzyme). This time was all about Node, and I can’t fall more in love with it.
• Destructuring: This is one of my favorite js features. This was not the first time I use it, but it was the time I had more fun time with it! In the following example, Im destructuring an array into const using the index and defaulting to some value 😵

• Travis install: I discovered that by default, Travis does a shallow clone, this means that is not fetching the whole GIT history.

git clone — depth=50 — branch=master https://github.com/zzarcon/run-when.git

This is fine 99% of the times, as you don’t care about that info, but in this case you actually need an up-to-date master to compare your changes with. You can achieve that doing:

Conclusion

I hope you all enjoyed reading the article just as much as I did playing with the tool! If you have any question or improvement please feel free to reach!

You can follow me on Twitter or Github @zzarcon