How To Speed Up Continuous Integration Build With New NPM CI And package-lock.json
While very controversial, the new npm release 5.7.0 brings some amazing features which will have noticeable positive impact on your development workflow!
From official changelog…
npm cicommand installs from your lock-file ONLY. If your
package.jsonand your lock-file are out of sync then it will report an error.It works by throwing away your
node_modulesand recreating it from scratch. Beyond guaranteeing you that you’ll only get what is in your lock-file it’s also much faster (2x-10x!) than
Generate and use package-lock.json file
I have hard time guessing what ratio of developer is using
package-lock.json already. It wasn’t behaving intuitively when first introduced and I suppose many people kept on deleting the file to prevent unnecessary headache.
Current behavior is luckily much more in line with what is expected by most developers. Running
npm install will generate
package-lock.json file if it didn’t exist with the versions from current
node_modules . Manual bumping of versions in the
package.json will result in correct version bumps in already existing
package-lock.json when using
On the other hand, newly introduced
npm ci ignores
package.json (only throws error if they are out of sync) and install dependencies as specified in
How to enable faster install in Travis CI environment
Travis CI is great CI server with seamless integration with public Github repositories so it’s a very popular choice among the OSS projects.
Travis CI runs couple of build steps for every triggered build. One of them is the install step which runs
npm install for all node projects by default. Luckily there is a simple way to override this default configuration.
We have to add
install section and use
npm ci instead.
As of 23. 2. 2018
npm v5.7.1is still in
pre-releaseso we will not get it by default in Travis CI environment and we have to install it manually. This step won’t be necessary in the future.
.travis.yml will look something like this…
Build time improvements
This configuration resulted in almost 3 times faster installation of dependencies saving more than 80 seconds for small to mid sized Angular CLI project.
Almost 3 times faster installation of dependencies achieved by small change in configuration is just amazing!
Similarly, fresh local install of checked out project now takes ~70 instead ~170 seconds!
Resulting builds are faster and the dependencies are guaranteed to be installed in a consistent way across different environments from local dev machines to CI servers.
What about Yarn ?
yarn install takes ~90 seconds on the same machine which is 20 seconds compared to running
npm ci .
Other CI servers
Lately I have been using mainly Travis CI but there are many other great CI servers like Jenkins, Teamcity, Bamboo and others which support wide range of flexible configurations.
In case you’re using other CI server, simply adjust step which is responsible for preparing node environment to use specific ( 5.7.1 ) npm version and
npm ci command to install dependencies.
What about reported security issues?
It seems to have completely broken my filesystem permissions and caused me to have to manually fix the permissions of critical files and folders.
This destroyed 3 production server after a single deploy!
Luckily this behavior was fixed by following
5.7.1 release with official response…
This release reverts a patch that could cause some ownership changes on system files when running from some directories when also using
So please, proceed with caution and preferably test in non-critical environments first.
What if I am using Travis CI caching for node_modules folder?
Travis CI can preserve whole
node_modules folder between the builds. Every subsequent build then performs just incremental changes on top of that during
npm install which may lead to even faster builds on some projects.
npm ci always deletes and re-fetches all dependencies as specified in
package-lock.json . This should in theory prevent any possible corruption of the already installed dependencies.
As always, use what makes most sense for your particular situation
And we’re done!
I hope you will use this tip to get much shorter build times for your projects too! Please support this article with your 👏👏👏 to help it to reach wider audience!
Also, feel free to check some other interesting frontend & Angular posts…
How to get better type checking and code completion support with Typescript in Angular Ngrx reducers or any other…medium.com
Learn how to create, publish and use your own custom Angular Schematics to increase your productivity and enhance…medium.com
Slow internet is a fact of life in many places around the world. Prompt users to wait instead of leave with nice…medium.com
Managing multiple requests with RxJS can be done in multiple ways. Each with its pros & cons. Learn when to use…blog.angularindepth.com
And never forget, future is bright