Spike, for Roots Users

a guide for our amazing community

With the public release of spike, we will also be putting roots into maintenance mode, with end-of-life one year later. Spike will be able to produce anything that roots was able to, and do it more quickly and efficiently. Internally, we have already switched over all new client work to spike, and pushed a number of sites to production that are performing well. In fact, we have been doing this since the alpha stages of spike’s development.

As such, it’s strongly recommended that roots users should consider spike for new projects. That being said, there is a lot that’s different between the two projects (which is why spike is a different project rather than a new version of roots), and we’ll go over it all in detail in this article.

Plugin-Based Architecture

First I want to discuss a high-level change that runs throughout spike, but was not at all present in roots. Static site generators are all about transforming html, css, and javascript. If you don’t need to transform these languages, you don’t need a static site generator, you can just write your site directly to the files you will use for production. But there are a lot of helpful tools like babel and autoprefixer that make building websites much nicer, and as such many developers like having the ability to transform.

Roots was built on top of accord, an engine which was able to transform a large range of html, css, and js projects, and normalize their inputs and outputs so that roots was able to treat any language the same way and users could seamlessly switch between them without needing to rewrite their configuration. Accord worked kind of like this:

Spike accomplishes the same goal, but in a completely different way. Rather than taking and entire language-transforming project and normalizing it through something like accord, Spike is built on libraries which parse html, css, and javascript, expose the parse trees for plugins to modify, and return the results. So it looks more like this:

Both methods accomplish the same goal — allowing users to transform html, css, and javascript in a variety of ways. But in our opinion, the way that Spike does it is significantly better. With plugin-based architecture, users gain a couple significant advantages:

• The ability to use only exactly the features they want
• The ability to understand and contribute to pieces of their architecture without needing to grok monolithic codebases
• The ability to build and add their own transformation features quickly and independently without needing to make a pull request to another large project

For example, if a user wanted to add a feature to jade/pug, even beginning to make an attempt at this is a seriously daunting process. A good place to start might be the lexer, which is a whopping 1300 LOC, and must be contextualized alongside the parser and code generator, another two enormous projects (~2000 more LOC), in order to really come together. It’s not wonder jade has trouble picking up new contributors.

The issue is not that jade is poorly written — on the contrary, it’s a fantastic, battle-tested project used by many thousands every day. The issue is that all the functionality is bundled into one large fixed-spec project. Jade/pug has a very long list of features, all of which are fantastic, but also all of which are handled within that one codebase.

When working on reshape, spike’s foundation library for transforming html, an explicit goal was to make sure that each feature was split into its smallest logical unit, and that the code was accessible for contributors. For example, take a look at the sugarml lexer, which is about 1/4 of the size at 308 LOC, even when you count the much more extensive comments in the source. At the same time, sugarml only does a small fraction of what jade/pug does. It’s simply whitespace-based parsing. No loops, no conditionals, no mixins, no includes, no layouts, etc . Those are handled by separate libraries which are equally small and easy to understand, and can be included by users only if and when they want.

A Move Towards Standards

When we started building roots a few years ago, IE7 was still on our necessary browser support list, and ES6 was a term nobody had ever heard of. But today, browsers and web-based languages are improving rapidly, which calls for a change in philosophy.

In roots’ default stack, every language was based on the language author’s arbitrary thoughts and preferences as to what might be easier and more convenient to read, write, and/or understand. But now, web standards committees are becoming much more active in building official specifications for the future or html, css, and javascript. This means that we have a more “official” reference, and one that might eventually not even need to be transformed at all when the standard is supported widely enough. These official standards are created by selective committees of very technically capable people, often including representatives from the companies that build the major browsers, and are widely accepted by the community. As such, it’s an easy decision for Spike to move towards support for the official specifications.

Many readers are likely familiar with ES6, the next generation of javascript. It has come so far, so fast, and the progress has been incredible. In fact, the latest stable version of node.js fully supports ES6, and Spike itself and all its plugins and related libraries are written directly in ES6 with no compilation or transformation. And it feels great. While browsers will take a little longer to catch up, the ability to write ES6 directly for node projects with no compile step is just a little taste of the future we will soon have. With the exception of IE11, which is on the way to end-of-life, all major browsers are now “evergreen”, yes, even microsoft’s browser, which means the future will see accelerated adoption of new specs and standards. Yes indeed my friends, it’s a good time to be a web developer.

So with that being said, let’s take a look at the new stack of plugin-based foundation libraries that spike uses to transform your code, how they differ from roots’ default options, and the new standards that they follow.

Stylus -> Postcss

While roots is flexible as to which CSS preprocessor you can use, it defaults to stylus, and this is the choice of many of roots’ users. In Spike, we are dropping stylus completely in favor of postcss. That being said, you can build up a set of features very similar to those that stylus has using postcss if you want.

However, we are recommending and including cssnext in the default stack, which is a bundle of postcss plugins that allow developers to utilize features that are slated for future versions of CSS. These features come from the CSS Working Group, a group that produces standardized documentation for CSS features to be included by browsers. The hope is that over time, cssnext will shrink and perhaps eventually be entirely unnecessary as browser vendors implement the CSS Working Group’s specifications. But for now, it is necessary.

Cssnext is not the only thing you can use with Spike. Postcss is simply a platform for plugins, and there are huge numbers of plugins already available. While we recommend certain plugins in spike standards, users are welcome to customize as they wish.

Coffeescript -> Babel

Coffeescript was a fantastic language and paved the way for many of the features that ended up in ES6, but it is now time to move forward. Spike’s default stack is built on babel’s ES2016 transform, meaning any proposals compliant with the ECMA262 javascript spec. This will eventually be supported by all browsers, which is very exciting.

While most people equate babel directly with transforming javascript to ES6, it’s actually much more powerful than that. Like all of our other foundation libraries, Babel is simply a platform for plugins that transforms the javascript AST. As such, there is no type of future update to javascript or tool you can build for javascript that can’t be handled by babel.

As it stands, we plan on updating the default stack each year, as TC39 releases new proposals that have reached stage 4. However, users are welcome to use any plugins they would like and will not be forced to upgrade at any time. You can always find an up-to-date listing of our spike standards here.

Jade -> Reshape

You probably won’t be surprised at this point to hear about our plans for html. Reshape is, much like babel and postcss, a foundation-level base for plugins to transform html. Reshape is the newest of these three libraries, and was built by the static-dev team, so we will be actively maintaining it (although we also welcome other interested maintainers to join).

While the syntax of our html standards remains very close to what jade looked like, there are several important differences. Our hope with this stack is to maintain the cleanliness and efficiency of jade while making each feature optional, and the core code much more accessible to contributors.

As expected, this is based on the work of WHATWG, a HTML standards group. Reshape’s core parser is built using parse5, which is maintained by WHATWG members. We are thinking heavily about http/2 and how we will need to adapt our methods once we start switching over all our servers (which will start happening very soon, as http/2 is supported in all major browsers, and is over 75% global usage). These changes and modifications will be reflected in our spike html standards as they develop and are tested internally.

Js Pipeline -> Webpack

Javascript bundling is an extremely important feature for developers, and while it is not exactly in line with the other three libraries here in that it’s not a parse, plugins, generate flow, it does merit discussion.

In roots, our default was the js pipeline, although many people preferred using browserify. Essentially, we left the bundler entirely up to the user. In spike, this is not the case — the bundler is webpack. In fact, webpack handles the compilation of all files, html, css, and javascript.

The good news is that webpack also takes plugins, which can be used to modify the manner in which any given file is compiled or bundled. In fact, spike’s core is mostly just plugins that manage the compilation and output of the html, css, and static asset files.

Upgrade Path

Phew, that was a lot. So you might be asking — what is the upgrade path for roots sites to spike? Honestly, there isn’t one. There are so many changes between the two pieces of software that any type of migration would be extremely time-intensive and difficult. Again, this is not a new version of roots, it’s an entirely different project.

IMHO, the better option for roots users is to look at spike for any new projects you might be creating. We have very thorough documentation on how to use spike, and hopefully the higher-level concepts have been covered well enough in this article.

And as always, if you need help, you can join us in our gitter channel, and if you are in need of more dedicated support or partnership, you can always reach out to spike’s primary sponsor, Carrot.

If you liked this article, click the little green heart below to show your support. Thank you!