Refactoring an old and broken library to using async/await with TDD
In 2010 I released a small library (github) for Nodejs, for the bit.ly API. It’s been moderately successful — at the time of publishing this article it’s gained 19,049 downloads in the month of October 2017 and has 24 other libraries that depend upon it.
With that in mind, there had been some outstanding issues for around a year or so and recently I had completed an agile TDD and refactoring course through work — so I thought it would be a good time to apply the two.
In this article I’m not going to teach TDD itself (you can find resources like this online) but I’m going to show you how I set up an environment that provided continuous testing and integration while I refactored a library that hadn’t been touched in 2 years up to node 8 support, meanwhile providing fallback support to node 4 to ensure it didn’t break those 19,000+ users programs.
Old code smells
The old ES6 class was my first attempt at building a es6module that was backwards compatible with node 4. However as you can see it’s full of smells and repetition and with the issues reported clearly didn’t work. It was already set up on TravisCI, however the tests were failing and I hadn’t looked at them.
When I built npm-lint I had a simple yet effective Typescript compiler setup through webpack that I repeated here, but instead I would pass the es7Javascript through it and compile support back to es5, which means node 4 and 6 can still be supported.
The first task was to start writing some tests, and choosing a better framework for async/await tests. In the end I chose mocha-webpack as it allowed me to run es7 tests across all node versions by transpiling them first along with chai expects library. I also added sepia to the tests so that requests are cached and speed up text execution locally.
The test went through some changes but allowed me to find the bits of code that could be separated into two main categories — those that formed part of the library, and those that allow me to expose the public API for backwards compatibility. In the end the library test look like this.
This allowed me to ensure that as I refactored the library code that I could expect the same result. In the end it allowed me to create an async/await version of the method and I knew generateUrlwould return the correct values.
Once that was in place then I wanted to make sure the async/await code could be distributed to support different versions.
One thing that did change with this release is the removal of the new keyword when evoking the factory, and some endpoints were removed as bit.ly no longer supports them.
Webpack, Babel and Typescript to the rescue
It might seem like an odd choice to use Typescript for the transpiler for my es7 code, but the trick is in the config:
By targeting es5 and commonjs, and using allowJs we can pass in out Javascript, and as TypeScript is just a superset it easy compiles down to our target format.
As it uses babel-polyfill and target es5 we also get Promise available to node 4. To ensure that this works before a release happens I needed to set up a pipeline to check it.
This then uses the webpack config to output a distribution file that works across all the node versions. Eventually node 4 will be deprecated but I wanted to ensure during version 5.x of my library that I continue to support it.
Continuous Integration and Delivery
I’ve never really been one to write tests, but after writing a nice set of tests for this code I decided it really should be run within a CI and that it should also handle building and distribution of the library so I don’t forget.
I’ve used CircleCI before and found their services pretty easy to use, and with a good uptime and speed. After some initial work learning how their new workflows feature worked I created a set of jobs in a pipeline that work in tandem.
You can view the config here, it’s rather big
The first task is to run npm install. Actually we first check for cached resources for the current version that we’re working with and upgrade any modules and install.
Next comes the interesting part, by only specifying that our test jobs need 1 task to continue, our install task we set them up to run in parallel. Because our code has no native modules we can easily run our tests across them using mocha-webpack to compile the tests for earlier versions.
When all those tests pass we trigger the webpack build phase, and finally we run the distribution. I have two tasks here, one for production based on the master branch and one for beta releases on any other branch. These also push the tags back to git after npm release.
To do this you need to add your NPM auth token and Git ssh push keys into CircleCI, but then it provides a push back to ensure your versions are up to date.
With this setup, you can support many more users and ensure that your code is well tested (and don’t disappoint those 19,000+ users!).
Next Steps
My next steps will be to build API documentation from the JSDoc comments and link it from the Readme, also I want to move to only release on specific branches such as beta . I also need to add support for a environment variable that switches between patch, minor and major when doing version bumping in the release step.
Hopefully this template can help you set up a happy continuous integration and deployment environment that ensures your software quality is high.
(Oh yea, TDD doesn’t suck)
