Modern Front End: The tools and build process explained

Motivation For This Post

Starting, or even understanding modern front end development can be difficult and confusing. If you are trying to spin up a personal project or launching a new page for your company and don’t have experience working with front end tools, figuring out where to start can seem daunting. You can’t really figure out each tools’ responsibility and function in the build process without doing a lot of googling and even then you need some base level understanding of what the tool is, what it does, and WHY it’s needed. Most posts I’ve read miss the WHY, and without understanding the why behind a tool and what the tool actually does, the front end build process seems confusing, annoying, or maybe even pointless.

So, the purpose of this article is to provide the why and what behind each build step and list some available tools to facilitate each process.

The target audience for this is anyone who is just getting into front end development and needs a place to start, or a back end developer looking for some insight into the madness.

Why is Front End “Built” Now?

Without giving a history lesson on how front end has changed over years, I think the easiest way to explain “why” is to address each build step individually and go into the problems they solve.

The different steps:

• Transpiling
• Bundling
• Minifying
• Packaging

Transpiling

Original Problem: Browser compatibility with advancements in JavaScript and CSS

JavaScript is changing and evolving faster than browsers can update and support. With the introduction of ECMAScript standard and frequent language updates, there has to be a way to convert the newest version of JavaScript to an older, compatible version. Otherwise, all the new hotness will go to waste and devs won’t be able to take advantage of stuff like arrow functions.

In addition to new versions of ECMAScript, there are many different flavors(i.e. CoffeeScript, TypeScript, Etc.). They all transpile back to vanilla JavaScript, resulting in backwards compatible browser support.

There’s also the advancements around CSS. The extension SCSS, which grants amazing syntactical sugar to CSS, has to be transpiled to CSS for browsers to apply styles to the DOM.

In short, transpiling lets developers take advantage of the newest, hottest updates to languages and extensions and maintain browser compatibility.

Here’s an example of what transpiling actually does: Arrow Functions

// arrow function syntax in array map
const double = [1,2,3].map((num) => num * 2);
// after being transpiled
const double = [1, 2, 3].map(function (num) {
    return num * 2;
});

Bundling

Original Problem:

1. Importing via Script tags isn’t scalable or manageable
2. New packages become available daily
3. New patterns and frameworks rely heavily on modularizing JavaScript

Per points 1 and 2: As more and more JavaScript packages came to be, the script tag region of index.html became unmanageable. As packages are updated and new versions are released, those script tags need updated. And if some of your imported packages relied on other packages, and those packages had cross dependencies, the script tag ordering and versioning gets messy.

To alleviate this, package managers were made: See npm

Package managers handle all the hassle of making sure dependencies are included, packages/libraries are up to date, and in some cases do a bit more.

Per point 3: In an effort to make front end code less spaghetti-like, more readable, and maintainable, modularizing has become the standard. Many patterns and modern frameworks, such as React, require it. The core concept is that smaller chunks of code are easier to understand, consume, and update for future requirements. These code chunks can also be shared and reused within the site or through npm for 3rd parties to use.

So now instead of using Script tags and having libraries be globally available, we are importing specific libraries into our modularized files. How do we turn this into a deliverable, browser compatible JavaScript file? Well, by Bundling of course.

Bundling is the process of taking all the “import” or “require” statements, finding the matching JavaScript snippets/packages/libraries, adding them to the appropriate scope, and packaging it all into one big JavaScript file. That file will be one of the few, if not only, script tags in your index.html file.

Minifying or Obfuscating

Original Problem: Source file sizes growing larger, load time becoming slower

With the large amount of packages and libraries being included, the final deliverable started ballooning in size. Loading up the script for the page took too long, and it was completely human readable, which makes for low-effort exploits. Hence, minifying and obfuscating.

Minifying reduces our final file size by removing white-space and code comments. You can take it a step further by adding an obfuscate step, which changes variable names, method names, obscuring our code so it is less human readable once delivered to the client.

Note: this helps with security by drastically increasing the effort required to understand the code, but shouldn’t be viewed as the security measure.

Here’s an example of minifying the code example from above:

// before minifying
const double = [1, 2, 3].map(function (num) {
    return num * 2;
});
// after minifying
const double=[1,2,3].map(function(num){return num*2})

Obfuscating would result in an even shorter result as it will change the double variable name to something as simple as d.

Packaging: Specifying Output Path

Original Problem: We have to put the bundled file somewhere

After all the above steps have finished, the compatible, bundled, minified/obfuscated file has to be put somewhere. The build tool handling the coordination of each step, i.e. webpack, will take the final result and put it where you specify. That specification is done in a config file, and is generally something like ./dist/app.js or ./dist/index.js. If you are putting SCSS and/or HTML through the build process you may have to specify the output of those files as well, or a general output location such as the entire./dist/ directory. If that’s confusing I think the example below will clear things up.

Common Tools and Example

Time to pull it all together and show it in action.

First, here’s a list of some commonly used tools to accomplish each step we discussed above, also including a couple options for package managers:

• Package Manager: npm, bower
• Transpiling: babel
• Bundling: webpack, gulp, Browserify, Parcel.js, Rollup.js
• Minifying: UglifyJS, minify, Closure Compiler
• Packaging: see bundlers

One thing to note is that these steps are generally all coordinated by the chosen bundler. As shown in the example below, the bundler will have a config available to specify the desired transpiler, minifier/obfuscater, and the name and destination of the output file(s).

With that, let’s dig into an example!

For this example I’m using the following tools:

• Package Manager: npm
• Transpiler: babel
• Minifier: UglifyJS
• Bundler: webpack

First, you’ll want to install node, which installs npm with it.

Once you have node and npm installed, you’ll want to navigate to your project and run npm init in the root of where you put your front end code. This command will take you through some basic questions that determine how your package.json file will be configured initially. You can always go back and change it later, so don’t stress over your answers.

After you have npm all setup and your package.json file made, you’ll want to install webpack, babel, and UglifyJS as dependencies via the following commands:

• npm install --save-dev webpack
• npm install --save-dev webpack-cli
• npm install --save-dev babel-core
• npm install --save-dev babel-loader
• npm install --save-dev babel-preset-env
• npm install --save-dev babel-preset-react
• npm install --save-dev uglifyjs-webpack-plugin

Note: I use --save-dev instead of --save for these dependencies because they are only needed during build time and not needed once fed up to the client. You’ll want to use --save for dependencies still required after build.

Instead of installing UglifyJS directly, we want to use the webpack plugin. Behind the scenes the plugin will use the latest version of UglifyJS, which is UglifyJS3.

Next, you’ll want to create your webpack.config.js file which you can see our example of below:

// the path plugin helps resolve the root our of front end directory
const path = require('path')
// we need to import uglify to specify our minifier
const UglifyJsPlugin = require('uglifyjs-webpack-plugin')
// this is our webpack.config.js definition
module.exports = {
    entry: './src/index.jsx', // notes #1 & 2
    output: {
        path: path.resolve(__dirname, 'dist'), // note #3
        filename: 'app.bundle.js'
    },
    module: {
        rules: [
            {
                test: /\.(js|jsx)$/, // note #2
                exclude: /node_modules/, // note #4
                use: {
                    loader: "babel-loader" // note #4
                }
            }
        ]
    },
    plugins: [
        new UglifyJsPlugin() // note #5
    ],
    resolve: {
       extensions: ['.js', '.jsx'] // note #2
    }
};

Notes:

1. Here I specify the entry point for webpack. This means that webpack will first go to the index.jsx file, find any import or require statements, navigate into them and resolve dependencies. This keeps going down the chain until all imports are resolved and bundled.
2. This particular project was written in React, so I used .jsx files. Webpack and babel don’t automatically know to resolve .jsx files by default, so I need to tell both babel (in my rules declaration) and webpack (in my resolve declaration) to accept .jsx files as if they were .js files.
3. Here I specify that I want to output the result of the build process to land in a file name app.bundle.js into a directory named dist.
4. Specifying I want to use the plugin `babel-loader`. Webpack expects a loader, and the babel loader passes the files matching the specified pattern through the babel transpiler. We want to exclude node modules so we don’t transpile every included package as they should already be browser compat.
5. Lastly, we specify our minifier by passing it in as a plugin. For more customization on the minifier you can visit it’s documentation. There are some configurations to specify what to minify, what not to minify, etc.

Okay, so we have our Bundler configured and ready to go. Next we need to configure babel so it knows exactly what it’s transpiling from. Below you’ll see our babel config (.babelrc) file:

{
    "presets": ["react", "env"]
}

What this is telling babel is that it needs to use it’s predefined configurations of react and env to transpile the input down to the most compatible version of JavaScript. These presets need to be included in our dev dependencies via npm. You’ll see them included in our example package.json file below.

Great. Now we are passing our special .jsx files through our transpiler to make them browser compatible, and bundling them. But how do we kick off the build process?

For that we’ll use npm. Npm has two functions that it serves: 1) as a package manager and 2) as a task runner. When we setup our project we ran npm init to create our package.json file. When we add dependencies, like npm install react --save-dev our dependency is saved in that file. Also in package.json is a section called scripts. This is where we can define scripts to kick start the whole build process.

Below you’ll see our updated package.json file that shows all our installed dependencies, some I installed on my own, as well as our scripts that kick start our build processes:

{
    "name": "AppName",
    "version": "1.0.0",
    "description": "Short description of your app",
    "main": "app.js",
    "scripts": { // note #1
        "dev": "webpack-dev-server --mode development --open",
        "build": "webpack --mode production",
        "start": "webpack-dev-server --open"
    },
    "author": "Trevor Poppen",
    "license": "MIT",
    "devDependencies": {
        "babel-core": "^6.26.0", // note #2
        "babel-loader": "^7.1.4",
        "babel-preset-env": "^1.6.1",
        "babel-preset-react": "^6.24.1",
        "webpack": "^4.4.1", // note #3
        "webpack-cli": "^2.0.13",
        "webpack-dev-server": "^3.1.1", // note #4
        "uglifyjs-webpack-plugin": "^0.4.6"
    },
    "dependencies": {
        "lodash": "^4.17.5",
        "react": "^16.3.0",
        "react-dom": "^16.3.0",
        "react-redux": "^5.0.7",
        "redux": "^3.7.2"
    }
}

Notes:

1. Our scripts definition here let us run these commands in a shell via commands such as npm run dev or npm run start.
2. These various babel dependencies are all required for babel to properly take in and transpile our JavaScript. Your babel-preset-* dependencies will vary depending on what version or flavor of JavaScript you’re using.
3. We need to include webpack and webpack-cli for running our build steps.
4. webpack-dev-server is one method of hosting your site locally, say on localhost:8080 and seeing your changes on save and refresh. You can use other tools for local hosting as well, this is just the tool I went with since it was so simple.

Conclusion

Well, there you have it. A short summary on what each build step is, why it exists, and one example on how to setup your own front end build process.

Now that you know the why behind the madness, you can find your own preferences for tools, unique configurations, and what best suits your or your projects’ needs.

Further, you can look into some other “niceties” that aren’t a critical step in the front end build process, but improve the front end dev experience. Such tools could be Linters (syntax enforcers/helpers) and Reloaders (auto-refreshes your browser on file change).

I hope this article gave you some insight and a base level of understanding into the tools and purposes they serve, and hopefully cleared up some of the confusion. Let me know your thoughts or feedback in the comments, always looking to learn some things myself. Thanks!