Gulp and browserify-shim

faster builds and more tooling

4 min readApr 21, 2014

The tooling around Gulp and browserify is constantly evolving and improving, so it can sometimes be hard to keep up with best practices. Here’s a brief overview of recent changes, and how I’m currently approaching it. If you’re new to either gulp or browserify, I’d recommend starting with this article by Dan Tello.

Changes at a Glance

browserify-shim moves to JSON

Somewhat recently, browserify-shim has moved its configuration into the package.json. This works well for the browserify command-line tool, but isn’t necessarily as easy to integrate with Gulp and Grunt. Furthermore, manually editing and maintaining vendor paths in JSON can be pretty nasty.

gulp-browserify is blacklisted

The gulp-browserify plugin has been blacklisted, and instead we’re now encouraged to use something like vinyl-source-stream to access the browserify API directly. So, now our browserify gulp tasks look like this:

var source = require('vinyl-source-stream');
var browserify = require('browserify');//the core bundle for our application 
gulp.task('browserify', function() {
   return browserify('src/index.js')
      .bundle()
      .pipe(source('bundle.js'))
      .pipe(gulp.dest('app/js'));
});

I’m not a fan of the added boilerplate, but it does give us more flexibility and also ensures we are always working with the latest and most complete browserify API.

Fast and Maintainable Builds

Here are some things I’ve found when using browserify-shim for applications:

A lot of the libraries that we might need to shim are already on Bower. e.g. Three.js, Pixi.js, TweenLite, and hundreds more.
Where possible, we want to avoid manually editing or configuring paths for our dependencies. As our dependencies change and grow, this becomes a maintenance problem.
Often our shims are large files that weren’t designed with the Unix or NPM philosophy of “many small modules.” Examples might be Three.js and even jQuery. Pulling these packages out of the core bundle drastically improves our build time.
Eventually, we may want to distribute our application using CDNs for jQuery, Three.JS, etc. However, during development, this can also lead to slower reload times compared to fetching local files, and won’t work if you don’t have an internet connection (e.g. on a train to work).

With these factors in mind, my current approach is to sourcemap-concat all of my vendor libraries into one file, and use the “global” feature of browserify-shim to expose them with my desired aliases.

var concat = require('gulp-concat-sourcemap');//the third-party libs like Three.js, TweenLite, etc.
gulp.task('libs', function() {
   gulp.src( ... list of vendor libs ... )
      .pipe(concat('libs.js'))
      .pipe(gulp.dest('app/js'));
});

My HTML then only has two script tags: one for the libs.js, and another for bundle.js as we described earlier. I also need to expose each of the vendor modules for browserify-shim, using the “global” feature. The following is added to package.json:

 "browserify": {
     "transform": [
         "browserify-shim"
     ]
 },
 "browserify-shim": {
     "three": "global:THREE",
     "tweenlite": "global:TweenLite",
 }

As development becomes more stable later on, I can easily swap out Three.js and jQuery to fetch from their respective CDNs.

Dependency Discovery

As I mentioned, we should try to get away from manually editing and maintaining paths to vendor libraries. Bower paths do not follow a strict pattern, and they may change unpredictably as we update our Bower dependencies. Fortunately, most Bower packages point directly to the main JS files that we need to shim. So we can use a module like wiredep to determine the JS file paths of our Bower dependencies. So now our concat task looks like this:

var concat = require('gulp-concat-sourcemap');//grab the bower paths
var bower = require('wiredep')({
    //options, like ignores...
}); //concat all bower JS files
gulp.task('libs', function() {
    gulp.src( bower.js )
       .pipe(concat('libs.js'))
       .pipe(gulp.dest('app/js'));
});

Overrides

Some libraries, like GSAP’s TweenMax, may not reference the exact file you were hoping for in their bower.json’s “main” field. Or, they may not have a “main” field at all, which will lead to warnings when you try to use wiredep. The solution is to provide an override in your module’s bower.json. Here is an example where only TweenLite and TimelineLite are included in the bundle, instead of the default TweenMax.

 "overrides": {
    "gsap": {
       "main": [
          "./src/uncompressed/TweenLite.js",
          "./src/uncompressed/TimelineLite.js"
       ]
    }
 }

Unfortunately, we still need to manually edit the package.json to setup aliases, exports, and any additional shim paths that you may want to include in your bundle. Enter “shimbro” …

shimbro

https://www.npmjs.org/package/shimbro

shimbro (“Shim Bro”) is a little tool I developed more as a proof-of-concept than anything. So far, it seems to be working well in common scenarios. It allows you to quickly setup your browserify-shim aliases, paths, and exports through a command-line prompt.

In the above example, shimbro walks through our bower dependencies and wires up bower paths for us in the package.json. This tool lets us manage Bower dependencies and shims without having to think about paths or touch any JSON.

There are lots of things that the tool could explore, like an API for build integration, or supporting Component.io dependencies. The goal is to automate as much as we can, keeping our builds and non-NPM dependencies easy to maintain and update.

If you’ve found other techniques that work well with Gulp and browserify-shim, I’d be interested in hearing them.

Addendum

More recently, I’ve been using watchify for fast incremental bundling. Check it out:
https://github.com/substack/watchify