From Grunt to NPM — Part 1
Background, Motivation and Prepping
There is a lot of buzz around using NPM scripts as a task runner driving build processes, but most of the content out there speaks in slogans without really touching real project aspects and challenges, so we at Capriza decided to really dive into it and see if our Designer product’s front-end build process can benefit from it.
Here is what we’ve learned…
Capriza’s Designer is the tool to use in order to create your own Zapp (Micro-App) by dragging & dropping web components onto a mobile canvas while configuring the Zapp flows and the styling you desire.
The Designer is a Firefox add-on (a.k.a extension) which holds a web content and as such it goes through a standard build processes web front-end project go these days, and in our case, using Grunt.
Hey, what’s wrong with Grunt?
Why would you use tools like Grunt/Gulp and the likes to begin with?
Well, these tools provide a way to automate and break a build process into well defined units with distinct outcomes and later on combines them together to assemble final product artifacts.
It allows us to configure the process and control its flow, and though Grunt delivered what was expected of it, working with it over time became somewhat of a maintenance burden. Here are some of the reasons behind it:
- We felt that Grunt and the likes act as a redundant mediator between us and the node package (or CLI) actually doing the work. In some cases, the Grunt plugins provided APIs which were not directly supported by the node packages, thus inviting us to unknowingly break their conventions.
- As part of its architecture, Grunt relies on plugins. These plugins are also installed via NPM and reside under the projects node_modules directory. We realized that for each task we wish our build to preform we need to install the Grunt plugin which internally depends on the actual node package that does the work, so for example, to precompile handlebars templates we need the handlebars node package and the Grunt plugin to enable it via Grunt. This quickly caused our node_modules directory to bloat and also prolonged our clean NPM install time.
- We thrive to reduce the configurations involved in automatic process like builds and tests. Removing Grunt and its dependencies from the stack seems to be a good step in that direction.
- Debugging Grunt is hard. Going from understanding what the Grunt plugin does into what the inner npm package does is tedious and sometimes can send you the wrong path.
These are the main reasons behind our desire to find a better solution for our build process.
Why then NPM?
There are quite a few option like Gulp and Broccoli, why then use NPM?
As far as we were concerned there were 2 main reasons why NPM scripts suited our needs best:
Using NPM scripts is simple and to the point. If you need to know what the script does you go to the node package documentation and there you have it. No need to read any Grunt plugin or any other 3rd party docs.
Working on front-end project there is a great probability that NPM is already involved in the process, even if you’re not using it as your task runner tool.
So was it on our case. It is already there and does not require additional installation or configuration, so if we can use it for more purposes — why not?
More Control. Less Errors.
Invoking the node packages directly via CLI or creating your own scripts and triggering them gives you more control on what’s being done and avoids unexpected errors which might happen due to misalignment between the Grunt plugin and its corresponding Node package.
Flexibility and Updates
Using NPM means you have the ability to practically utilize every node package out there for your needs, which cannot be said about Grunt for Grunt needs a plugin wrapper over that node package — a plugin that might not exist yet. Moreover, in many cases it is hard to keep up to date with the latest node package versions since it’s Grunt wrapper we depend on does not support it yet.
Prepping towards migration
Grunt used to manage the entire Capriza’s Designer build process. The build was consisted out of several tasks. Each one of these is a candidate to be migrated to NPM and we shall later see the set of challenges some presented, but our first task was to understand what Grunt currently does for us.
Here are the main ones:
- Code Concatenation — Reading the main HTML files and extracting the scripts tags blocks from them to produce a concatenated, sometimes minified, files. This process results in Prod-ready HTML and JS files.
- Pre-Compiling Templates —Capriza’s Designer utilize Handlebars templates and as part of the build these templates are being per-compiled to produce JS files which represent them for performance concerns.
- Compiling LESS — Capriza’s Designer uses LESS for better maintaining CSS. Our build compiles the LESS files into production-ready CSS files.
- Copying files — Any build process has it — moving files from one location to another to prepare a distribution directory ready to be deployed.
- Amazon S3 deployment — The Designer, like many other projects, is being deployed to Amazon’s S3. There is a Grunt task which was made of several sub-tasks to preform this rather complex deployment process.
- Triggering Jenkins job — As a part of the development cycle at Capriza, developers are often running an in-house tool called “Watchdog” for E2E testing (automation tests). This process can be triggered by Grunt on demand. Of course this task is dependent on a deployed version which we want our E2E test to work on.
- Unit Testing — Running unit-tests is a core aspect of Capriz’a Designer development process. We are using Karma as our runner and Grunt triggers it. When triggered these tests will run in the background while you code to make sure that no regression happens and TDD is made more smoothly.
- Watch — In order to speed-up development, files are being watched for changes in order to generate their updated compiled artifact or to run unit-tests in the background. This “Watch” process is initiated by Grunt.
- Dev Tools — Capriza also has a tool for making the developer day-to-day work easier. It consists of a simple server to serve artifact and several watch commands which monitor modification to the source code and generate new artifacts accordingly on the fly to provide immediate feedback (like compiling LESS). The tool utilizes Grunt tasks by calling them upon source code change.
In this part we’ve seen what were the motivations behind migrating our Designer’s build process to NPM and the set of challenges ahead of us.
Part 2 of this article will send us diving into the details behind core tasks described above, leaving nothing behind for Grunt.