Rails Asset Pipeline

The asset pipeline does three main things:

  1. Concatenate
  2. Minify
  3. Preprocess

Concatenation

Suppose you have three different CSS files:

* app
** assets
*** stylesheets
**** navbar.css
**** posts.css
**** footer.css

Normally, you’d have three separate <link> tags:

<html>
<head>
<link href="navbar.css" rel="stylesheet">
<link href="posts.css" rel="stylesheet">
<link href="footer.css" rel="stylesheet">
...
</head>
<body>
...
</body>
</html>

This would instruct your browser to make a separate HTTP request for each of the three files:

Requests over the network are slow. It’d be better if we could just combine them all into one file, and just make one request:

// application.css
/*
navbar.css styles here
...
*/
/*
posts.css styles here
...
*/
/*
footer.css styles here
...
*/

The Asset Pipeline does this for us. It applies to JavaScript files as well as CSS files.

Minification

Minification makes file sizes smaller my removing unnecessary things, like comments and white space. Smaller file size = mor speed!

https://cssminifier.com/

With JavaScript, in addition to removing comments and white space, minification makes further optimizations. Like shortening variable names.

https://javascript-minifier.com/

Preprocessing

Consider an example:

// navbar.scss.erb
nav {
li {
a {
color: <%= color %>;
}
}
}

Preprocessing happens from right to left. First the file would be passed through the ERB preprocessor, which would substitute in red for <%= color %>:

nav {
li {
a {
color: red;
}
}
}

Then the file would be passed through the Sass preprocessor. It would compile the SCSS into regular CSS:

nav li a {
color: red;
}

Manifests and directives

rails assets:precompile is the task that does the compilation (concatenation, minification, and preprocessing). When the task is run, Rails first looks at the files in the config.assets.precompile array. By default, this array includes application.js and application.css. Let’s consider application.js:

//= require jquery
//= require app
//= require_self
//= require_tree dogs
//= require_tree .
var foo = 'bar';

I’ve modified it a bit for the purpose of instruction.

Lines prefaced by //= are instructions for the compiler. They’re called directives because they’re directing the compiler, telling it what to do.

//= require jquery

Obviously the instruction is trying to tell the compiler to require the jQuery library. But how does the compiler know where the file is?

config.assets.paths is an array of file paths. It tells the compiler where to look. When I created an app and ran bundle install this is what mine looks like (you can check yours by running rails console and then Rails.application.config.assets.precompile):

Rails.application.config.assets.precompile = [
"/Users/adamzerner/code/blog/app/assets/config",
"/Users/adamzerner/code/blog/app/assets/images",
"/Users/adamzerner/code/blog/app/assets/javascripts",
"/Users/adamzerner/code/blog/app/assets/stylesheets",
"/Users/adamzerner/code/blog/vendor/assets/javascripts",
"/Users/adamzerner/code/blog/vendor/assets/stylesheets",
"/Users/adamzerner/code/blog/.rvm/gems/ruby-2.4.1/gems/jquery-rails-4.3.1/vendor/assets/javascripts",
"/Users/adamzerner/code/blog/.rvm/gems/ruby-2.4.1/gems/coffee-rails-4.2.1/lib/assets/javascripts",
"/Users/adamzerner/.rvm/gems/ruby-2.4.1/gems/actioncable-5.0.2/lib/assets/compiled",
"/Users/adamzerner/.rvm/gems/ruby-2.4.1/gems/turbolinks-source-5.0.0/lib/assets/javascripts"
]

It initially just includes everything in app/assets, lib/assets, and vendor/assets. Then when you run bundle install, your gems add their paths to the config.assets.paths array so that you could include their files too.

When the compiler comes across //= require jquery, it’s going to:

  • Look for /Users/adamzerner/code/blog/app/assets/config/jquery.js, and not find it.
  • Look for /Users/adamzerner/code/blog/app/assets/images/jquery.js, and not find it.
  • Look for /Users/adamzerner/code/blog/app/assets/javascripts/jquery.js, and not find it.
  • Look for /Users/adamzerner/code/blog/app/assets/stylesheets/jquery.js, and not find it.
  • Look for /Users/adamzerner/code/blog/vendor/assets/javascripts/jquery.js, and not find it.
  • Look for /Users/adamzerner/code/blog/vendor/assets/stylesheets/jquery.js, and not find it.
  • Look for /Users/adamzerner/code/blog/.rvm/gems/ruby-2.4.1/gems/jquery-rails-4.3.1/vendor/assets/javascripts/jquery.js … and find it!

Once found:

  • There’s no extension (eg. .coffee), so there won’t be any preprocessing.
  • config.assets.js_compressor = :uglifier, so the compiler will use Uglifier as its minifier and minify the file.
  • In application.js, it will replace //= require jquery with the minified contents of the file /Users/adamzerner/code/blog/.rvm/gems/ruby-2.4.1/gems/jquery-rails-4.3.1/vendor/assets/javascripts/jquery.js. Like so:
// minified jquery source code here
//= require app
//= require_self
//= require_tree dogs
//= require_tree .
var foo = 'bar';

//= require app

Similar to //= require jquery, //= require app is going to look for app.js (it adds the .js extension automatically) under all the file paths in config.assets.paths. Once found, it will replace //= require app with the minified contents of the file:

// minified jquery source code here
// minified app.js code here
//= require_self
//= require_tree dogs
//= require_tree .
var foo = 'bar';

//= require_self

This directive basically says: “place the code in this file here”. So in our example, application.js will look like this after the //= require_self directive is processed:

// minified jquery source code here
// minified app.js code here
var foo = 'bar';
//= require_tree dogs
//= require_tree .

//= require_tree dogs

Suppose we have the following file structure:

- app
-- assets
--- stylesheets
---- navbar.scss
---- posts.scss
---- footer.scss
--- dogs
---- golden_retrievers
----- alpha.css
----- alpha.js
---- beta.css
---- beta.js
- lib
-- assets
--- stylesheets
---- library.css
--- javascripts
---- library.js
- vendor
-- assets
--- stylesheets
---- other_library.css
--- javascripts
---- other_library.js

require_tree dogs would find the files:

  • app/assets/dogs/golden_retriever/alpha.css
  • app/assets/dogs/golden_retriever/alpa.js
  • app/assets/dogs/beta.css
  • app/assets/dogs/beta.js

require_tree dogs looks for a dogs directory in app/assets, lib/assets, vendor/assets, and the other paths in the config.assets.paths array. Once it finds a dogs directory (under app/assets in this case), it compiles all CSS and JS files in it, regardless of how deeply they’re located in subdirectories (even the files in app/assets/dogs/golden_retriever get compiled).

It does this in alphabetical order of the file names. Therefore, despite the fact that alpha.css and alpha.js are nested more deeply than beta.css and beta.js, the alphas are compiled first, and the betas second. That means that styles in beta.css would override those in alpha.css.

If two files have the same name, the one that is nested less deeply is compiled first. Therefore, app/assets/dogs/alpha.css would be compiled before app/assets/dogs/golden_retrievers/alpha.css.

After executing the require_tree dogs directive, our application.js would look like this:

// minified jquery source code here
// minified app.js code here
var foo = 'bar';
// minified alpha.js code here
// minified beta.js code here
//= require_tree .

require_directory

In contrast to require_tree, the require_directory directive wouldn’t look in subdirectories. Thus it would only find:

  • app/assets/dogs/beta.css
  • app/assets/dogs/beta.js

Not:

  • app/assets/dogs/golden_retriever/alpha.css
  • app/assets/dogs/golden_retriever/alpha.js

Because those aren’t direct children of app/assets/dogs.

require_tree .

require_tree . looks for every CSS and JS file in our load paths. Contrast that with require_tree dogs, which only looked for files in dogs directories. So then, require_tree . will find:

  • app/assets/stylesheets/navbar.scss
  • app/assets/stylesheets/posts.scss
  • app/assets/stylesheets/footer.scss
  • lib/assets/stylesheets/library.css
  • lib/assets/javascripts/library.js
  • vendor/assets/stylesheets/other_library.css
  • vendor/assets/stylesheets/other_library.js

Note that it wouldn’t find:

  • app/assets/dogs/golden_retriever/alpha.css
  • app/assets/dogs/golden_retriever/alpa.js
  • app/assets/dogs/beta.css
  • app/assets/dogs/beta.js

Because those have already been dealt with by require_tree dogs.

After executing require_tree ., the compiler will have finished. The final state of application.js will look like this:

// minified jquery source code here
// minified app.js code here
var foo = 'bar';
// minified alpha.js code here
// minified beta.js code here
// minified library.js code here
// minified other_library.js code here

Fingerprinting

When you run rails assets:precompile, it compiles application.js, and writes the result to public/assets/application.js. If you set config.assets.prefix = "/foo", it’d instead write the result to public/foo/application.js.

Actually, instead of writing the result to public/assets/application.js, it writes it to public/assets/application-908e25f4bf641868d8683022a5b624.js. The thing at the end is called a hash.

This is called fingerprinting. Just like our fingerprints uniquely identify us, the hash uniquely identifies the particular version of the file.

Don’t worry, Rails is smart, so <%= javascript_include_tag "application" %> still works — it compiles to <script src="public/assets/application-908e25f4bf641868d8683022a5b624.js"></script>.

Why perform this fingerprinting? Because it allows us to cache. Imagine if we didn’t fingerprint. Suppose the user makes a request. We respond with the HTML page:

<html>
<head>
<script src="app.js"></script>
...

Since we want to cache, we tell the browser to cache app.js. Now suppose that we make a change to app.js. Next time the user makes a request for the web page, he still gets the following response:

<html>
<head>
<script src="app.js"></script>
...

Since we had the browser cache app.js, it uses the cached version instead of going out to the server for the updated version. And so the user is stuck with an old version of app.js, unable to receive our updated version until his cache expires.

Attempting to cache with no fingerprinting — FAIL

With fingerprinting, our first version is app-1.js (using a short hash for simplicity). Suppose the browser caches app-1.js. Now if we update it, we change the hash to 2, and thus the filename to app-2.js. So now when the user requests the web page, the HTML file has <script src="app-2.js">. Since it only has app-1.js in its cache — not app-2.js — it goes out to the server for the updated version.

Attempting to cache with fingerprinting — SUCCESS

Production vs. development

Consider config.assets.resolve_with. It’s an array with three possible states:

# Dev where debug is true, or digests are disabled (dev default)
%i[ environment ]

# Dev where debug is false, or production with compile enabled.
%i[ manifest environment ]

# Production default.
%i[ manifest ]
  • %i [ manifest ] means “Serve precompiled assets. Ie. the ones in public/assets.”
  • %i [ environment ] means “Compile the assets on the fly and serve them as separate files.”
  • %i [ manifest environment ] means “First try to serve from public/assets. If you don’t find anything, then compile it on the fly and serve as a separate file.”

Why serve them as separate files in development? Because it’s a lot easier to debug this:

Separate files; no minification — easy to debug

Than this:

One big file; minified — hard to debug

Sass

If you’re using Sass, you probably don’t want to be using the Asset Pipeline. Why? Consider this situation:

// _variables.scss
$color: red;

// main.scss
h1 {
color: $color;
}

// application.scss
//= require _variables.scss
//= require main.scss

First _variables.scss is preprocessed and included:

// application.scss
$color: red;
//= require main.scss

Then main.scss is preprocessed and included:

// application.scss
$color: red;
h1 {
color: $color;
}

Finally, application.scss is written to public/assets/application.css and served to the client when requested. But the code above is invalid CSS. We would have needed the Sass preprocessor to run on application.scss itself to turn it into:

// application.scss
h1 {
color: red;
}

Unfortunately, this doesn’t happen. So we need to use the native Sass @import directive. The sass-rails gem customized Sass to look under app/assets, lib/assets, vendor/assets and the rest of the paths when trying to @import something.

Note: if you’re using bootstrap-sass, you can override variables by simply redefining the variable before the @import directive.

For example:

$navbar-default-bg: #312312;
$light-orange: #ff8c00;
$navbar-default-color: $light-orange;
@import "bootstrap"; 

The reason why this works is because Bootstrap uses !default, which tells assignments not to override the variable if already has a value.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.