Brief history of JavaScript Modules

Brick Modules and Dependencies

Are you new to JavaScript and confused about module, module loader and module bundler? Or you have been writing JavaScript code for a while, but can’t get a grip of the jargons about module? You heard about the jargons like CommonJS, AMD, Browserify, SystemJS, Webpack, JSPM, etc., but don’t understand why we need them?

I will try to explain what are they, and what problem they try to solve, and how they solve the problem.

Example Application

Successful application launch

In this post, I’ll use a simple web application to demonstrate the concept of module. The application displays the sum of array on the browser. This application consists of 4 functions and one index.html file.

Dependency diagram of the functions

The main function calculate the sum of numbers in the array and display the answer to span#answer. The sum function has dependency on two functions: add and reduce. Add function does what it says it does; it adds two numbers. Reduce functions will iterate through array and call iteratee call back function.

Take a time to understand the code below. I will use the same functions again and again.

Let’s look at how we can put these pieces together to build an app.

Inline Script

Inline script is when you add JavaScript code between <script></script> tag. This is how I started JavaScript. I believe most of JavaScript developer have done this at least once in their life time.

This is very good way to get started. There is no external files or dependency to worry about. But this is also perfect recipe for unmaintainable code because of following:

  • Lack of Code Reusability: If we need to add another page and need some of the functions from this page, we will have to copy and paste the code
  • Lack of Dependency Resolution: You are responsible for having add, reduce and sum functions come before main part of the script.
  • Pollution of global namespace: All the functions and variables will reside on global scope.

Script tags

This is natural transition from inline scripting. Now we divide the big chunk of JavaScript into smaller pieces and load them using <script src=”…”></script> tags.

By separating files into multiple JavaScript files, we can reuse the code. We no longer need to copy and paste code between different html pages. We simply need to include the file with script tags. Although this is better approach, this still has following problems.

  • Lack of Dependency Resolution: The order of the files are important. You are responsible for including add.js, reduce.js and sum.js files before main.js file.
  • Pollution of global namespace: All the functions and variables are still in global scope.

Module Object and IIFE(Module Pattern)

By using module object and Immediately-invoked function expression(IIFE), we can reduce global scope pollution. In this approach, we expose only one object to global scope. The single object contains all the methods and values we need in our application. In this example, we expose only myApp object to global scope. All the functions will be held onto myApp object.

Note that each file except my-app.js is now wrapped into IIFE format.

(function(){ /*... your code goes here ...*/ })();

By wrapping each file into IIFE, all the local variables stay within the the scope of the function. So all the variables in the functions will stay within the scope of the function without polluting the global scope.

We expose add, reduce and sum functions by attaching them to myApp object. And we access these functions by referring myApp objects like this:

myApp.add(1,2);
myApp.sum([1,2,3,4]);
myApp.reduce(add, value);

We can also pass myApp global object via IIFE parameter just like shown in ‘main.js’ file. By passing the object as parameter to IIFE, we can choose shorter alias for the object. And our code will be a little bit shorter.

(function(obj){
// obj is new veryLongNameOfGlobalObject
})(veryLongNameOfGloablObject);

This is huge improvement compared to previous example. And most of popular JavaScript libraries such as jQuery use this pattern. It exposes one global object, $, and all the functions are under $ object.

Yet, this is not a perfect solution. This approach still suffers from the same problem as previous section.

  • Lack of Dependency Resolution: The order of the files are still important. myApp.js file must come before any other files. And main.js file must come after all other library files.
  • Pollution of global namespace: Number of global variable is now 1, but is not zero yet.

CommonJS

In 2009, there were discussions about bringing JavaScript to server side. Thus ServerJS was born. ServerJS later changed its name to CommonJS.

CommonJS is not a JavaScript library. It is a standardization organization. It is like ECMA or W3C. ECMA defines the language specification for JavaScript. W3C defines JavaScript web API, such as DOM or DOM events. The goal of CommonJS is to define common APIs for web server, desktop and command line applications.

CommonJS also defines APIs for module. Since there is no HTML page and no <script> tag on server application, it make sense to have some clear APIs for modules. Modules need to be exposed(export) to others for use and be accessible(import). Its module syntax for export looks like this:

// add.js
module.exports = function add(a, b){
return a+b;
}

The above code is to define and export a module. This code is saved in add.js file.

To use or import add module, you need require function with filename or module name as parameter. The following syntax describes how to import a module to your code.

var add = require(‘./add’);

If you have written code on NodeJS, this syntax may look familiar. That’s because NodeJS implemented CommonJS style module API.

Asynchronous Module Definition(AMD)

The problem with CommonJs style module definition is that it is synchronous. When you call ‘var add=require(‘add’);’, the system will be on halt until the module is ready. This mean this line of code will freeze the browser while all the modules are being loaded. So this may not be the best way to define modules for browser side application.

To transfer module syntax from server usage to browser usage, CommonJS proposed several module formats, “Module/Transfer”. One of the proposals, “Module/Transfer/C” later become Asynchronous Module Definition (AMD).

AMD has following format:

define([‘add’, ‘reduce’], function(add, reduce){
return function(){...};
});

The define function(or keyword) takes list of dependencies and a callback function as arguments. The arguments for the callback function will be the same order of dependencies in the array. This is equivalent to importing modules. And the callback function returns a value, which is the value you export.

CommonJS and AMD solves the two remaining problems with module pattern: dependency resolution and pollution of global scope: We only need to take care of dependencies for each module or each file. And there is no pollution for global scope.

RequireJS

AMD can save us from the hell of script tags and global pollution in our browser application. Then, how can I use it? RequireJS is here to help us. RequireJS is a JavaScript module loader. It helps load modules asynchronously as needed.

Despite its name, its goal is not to support CommonJS ‘require’ syntax. With RequireJS, you can write AMD style module.

Before you write your own application, you will have to download ‘require.js’ file from RequireJS website. The following code is the example application written with RequireJS.

Example Application on AMD style

Notice that there is only one script tag in index.html file.

<script data-main=”main” src=”require.js”></script>

This tag loads ‘require.js’ library to the page. And ‘data-main’ attribute tells RequireJS to where is the starting point of your application. By default, it assumes that all files have ‘.js file extension. So it is ok to omit ‘.js’ file extension. After RequireJS loads ‘main.js’ file, it loads its dependencies and dependencies’ dependencies, etc. The browser’s developer tool shows all the files are loaded in following order.

Browser loaded ‘index.html’ and ‘index.html’ loaded ‘require.js’. Rest of the files and dependencies are all taken care by ‘require.js’.

RequireJS and AMD solve all the problems we had before. However, it creates other slightly less serious problems.

  • AMD syntax is too verbose. Since everything is wrapped in ‘define’ function, there are extra indentation for our code. With a small files, it is not much problem, but for a large code base, it can be mentally taxing.
  • List of dependencies in the array must match the list of arguments in the function. If there are many dependencies, it can be hard to maintain the order of the dependencies. If you have dozens of dependencies in your module, and later if you had to remove one in the middle, it can be difficult to find matching module and argument.
  • With current browsers(HTTP 1.1), loading many small files can degrade the performance.

Browserify

Because of these reasons, some people want to use CommonJS syntax, instead. But CommonJS syntax is for server and synchronous, right? Browserify is here to rescue you! With Browserify, you can use CommonJS module in browser application. Browserify is a module bundler. Browserify traverse the dependency tree of your codes and bundle them up in a single file.

Unlike RequireJS, Browserify is a command line tool. You will need NodeJS and NPM to install it. Once you have nodeJS installed on your system, type the following command.

npm install -g browserify

Let’s take a look at our example application in CommonJS syntax.

You may notice that in our ‘index.html’ file, the script tag loads bundle.js. Where is the bundle.js file? Browserify generate the file for us, once you execute following command.

$ browserify main.js -o bundle.js

Browserify parses main.js for require() function calls and traverses dependency tree of your projects. Then it bundle them up in a single file.

Here’s what ‘bundle.js’ file look like.

Browserify generated bundle.js

You don’t have to understand this bundle file line by line. But it is just worth note that all the familiar codes, main file and all of its dependencies, are included in this file.

UMD — Just to make you more confused

Now we’ve leaned about Global Object, CommonJS and AMD style modules. And there are libraries that can help us use either CommonJS or AMD. But what if you are writing a module and deploy to the wild internet? Which module style do you need to write?

Writing three different module types, i.e., global module object, CommonJS and AMD is an option. But then you will have to maintain three different files. And users will have to identify the type of the module they are downloading.

Universal Module Definition(UMD) is to solve this particular problem. Under the hood, UMD is a series of if/else statements to identify module style that current environment supports.

sum module written in UMD style

ES6 module syntax

JavaScript global module object, CommonJS, AMD and UMD. There are too many options. Now you may ask, what should I use for my next project? The answer is none of them.

JavaScript didn’t have module system built into the language. That’s the reason we have so many different way of importing and exporting modules. But that has recently changed. With ES6 language spec, module is part of JavaScript. So the answer to the question is you need to use ES6 module syntax if you want your project to be future-proof.

ES6 uses ‘import’ and ‘export’ keywords to import and export modules. Here’s the example application written in ES6 modules.

Example Application written in ES6 module syntax

There are many catch phrases about ES6 module: ES6 module syntax is concise. ES6 module will rule the JavaScript world. It is the future. But unfortunately, there is one problem. Browsers are not ready for this new syntax. At the moment of writing, only Chrome browser supports import statement. Even when most of browsers support ‘import’ and ‘export’, if your application has to support older browsers, you may run into a problem.

Fortunately, there are tools available. These tools allow us to use ES6 module syntax today.

Webpack

Webpack is a module bundler. Just like Browserify, it traverses dependency tree and bundles up into a single or more files. If it is the same as Browserify, why would we need yet another module bundler? Webpack can handle CommonJS, AMD and ES6 modules. And Webpack comes with more flexibility and cool features like:

  • Code Split: When you have multiple apps sharing same modules. Webpack can bundle your code into two or more files. For example, if you have two apps, app1 and app2, and both shares many modules. With Browserify, you would have app1.js and app2.js. And both contain all the dependency modules. But with Webpack, you can create app1.js, app2.js, and shared-lib.js. Yes, you will have to load 2 files from html page. But with hashed filename, browser cache and CDN, it can reduce initial loading time.
  • Loader: With custom loaders, you can load any file into your source. You can use ‘reuiqre()’ syntax to load not just JavaScript files, but also css, CoffeeScript, Sass, Less, HTML for template, images, etc.
  • Plugin: Webpack plugins manipulate your bundles before it is written into files. There are many community built plugins. For example, there are bundle for adding banners to bundled code, adding source map and splitting a bundle into chunks, and more.

WebpackDevServer is development server that automatically bundles your source code and refresh browser whenever source code changes detected. It will expedite your development process by providing instant feedback of your code.

Let’s take a look how we can build the example application with Webpack. Webpack requires a little bit of bootstrapping and configuration.

Since Webpack is JavaScript command line tool, you will need NodeJS and NPM installed. Once you have NPM installed, initialize project.

$ mkdir project; cd project
$ npm init -y
$ npm install -D webpack webpack-dev-server

You need a configuration file for webpack. At a bare minimum, you will need entry and output field for your configuration. Save following content in webpack.config.js file.

module.exports = {
entry: ‘./app/main.js’,
output: {
filename: ‘bundle.js’
}
}

And open your ‘package.json’ file and add following lines under ‘script’ field.

"scripts": {
"start": "webpack-dev-server -progress -colors",
"build": "webpack"
},

Now add all your JavaScript modules under ‘project/app’ directory and ‘index.html’ under ‘project’ directory .

Note that ‘add.js’ and ‘reduce.js’ is written in CommonJS style, while ‘sum.js’ is written in AMD. Webpack, by default, can handle CommonJS and AMD. If you have ES6 modules, you will have to install and configure ‘babel loader’ for that.

webpack dev server in action

Once you have all the files ready, you can run your application.

$ npm start

And open your browser and point url to http://localhost:8080/webpack-dev-server/.

At this point, you can open up your favorite editor and edit your code. As you save the file, the browser will automatically refresh to show the result of the change.

One thing you may notice is that you will not find ‘dist/bundle.js’ file. That’s because Wepback Dev Server creates bundle, but it does not write into the file system. It serves the bundled code just out of memory.

For deployment, You may want to create a bundled file. You can create ‘bundle.js’ file with following command.

$ npm run build

If you are interested in learning more about Webpack, visit Webpack document page.

Rollup (May 2015)

Have you ever include a large JavaScript library only to use a few of its functions? Rollup is another JavaScript ES6 module bundler. Unlike Browserify or Webpack, rollup only includes the codes that are used in your project. If you have a large module with many functions yet you only need a few of them, rollup will include only the functions you need into the bundle. This can reduce the size of bundle significantly.

Rollup can be used as a command line tool. Once you have NodeJS and NPM, install rollup with following command.

$ npm install -g rollup

Rollup can work with any type of module style. But, ES6 module style is recommended to take advantage of the tree-shaking feature. The following code is the example application written in ES6.

Note that I introduced another function, ‘sub()’, in add module. But this function is not used in anywhere in the application.

Let’s bundle this code with rollup.

$ rollup main.js -o bundle.js

This will generate ‘bundle.js’ file like following.

Note that ‘sub()’ function is not included in this bundle file.

SystemJS

SystemJS is a universal module loader. It loads the module dynamically on browser or NodeJS. And it supports CommonJS, AMD, global object and ES6 modules. With plugins, it can load not only JavaScripts, but CoffeeScript and TypeScript.

Another advantage of SystemJS is that it is built on top of ES6 module Loader polyfill. So its syntax and API will most likely be part of the language in the future. It will make your code more future-proof.

To import a module asynchronously with SystemJS, you can use following syntax

System.import(‘module-name’);

And you can configure the behavior of SystemJS with configuration API.

System.config({
transplier: ‘babel’,
baseURL: ‘/app’
});

The above configuration will let SystemJS use babel as transplier for ES6 modules and load modules from ‘/app’ directory.

As modern JavaScript application become bigger and more complicated, so does development workflow. So we need more than just module loader. We would have to find development server, module bundler for production and a package manager for third party modules.

JSPM

JSPM is swiss army knife of JavaScript development tool. JSPM is a (1)package manager, (2)module loader and (3)module bundler.

Modern JavaScript development rarely ends up with your own modules. Almost always, you will need third party modules.

With JSPM, you can install third pary modules from NPM or Github using the followin command:

jspm install npm:package-name or github:package/name

This will download the package from ‘npm’ or ‘github’ and install it on ‘jspm_packages’ directory.

While in development mode, you can use ‘jspm-server’. Just like Webpack Dev Sever, it detects the code change and reloads browser to show the changes. Unlike Webpack Dev Server, jspm-server make use of SystemJS module loader. So it doesn’t read all the files to bundle them every time it detects changes in a file. It just loads the modules that are required for the page you are working on.

When it comes to deployment, you would want to bundle your codes. JSPM comes with bundler. You can bundle your code with

jspm bundle main.js bundle.js

Under the hood, JSPM uses rollup for its bundler.

Conclusion

I hope this give you enough information for you to understand vocabularies about JavaScript modules. Now you may ask, what should I use for the next project? Unfortunately, I can’t answer that question. Now you have power to do your own research. I hope that it will be much easier for you to understand documents and articles about the tools I mentioned.

You can find all the code example in this Github repository. If you have any questions, please leave a comment below.