Consistency & The Fragmented Stack
and the solution: spike standards
Somewhere around five years ago, Ruby on Rails was all the rage. Thousands of new startups were basing their architecture on rails. Rails developers got boatloads of recruiting emails every day, the job market was starved. Rails had revolutionized web development, entirely. But why?
It made for a very fast zero to deploy time, it had strong consistent conventions, and a good developer interface. Nothing like this had existed before. The previous norm was cobbling together your own server using PHP. Today, I want talk about the conventions specifically. Rails was known for it’s strong adherence to convention over configuration. The advantages that come with this are fantastic. A rails developer could jump into any project built with rails, and be entirely at home. All projects were the same, because of the strong conventions. But while this was a great advantage, it ultimately ended up being a large part of rails’ downfall.
A couple years later, Rails started to fall off. Within Carrot, we were starting to move off of rails for an alternate approach — the split api/client stack. This approach offers many benefits, which you can read more about in the post linked above. One of the most important advantages is more flexibility. Because of its strong conventions, Rails was almost entirely inflexible. All code had to be in ruby. The client was strongly coupled to the server. Any internal changes to fit a specific project were somewhere on the spectrum from extremely difficult to impossible. As soon as we started running split-client projects, we felt the freedom of flexibility, and it felt great. APIs were small, focused, simple, and entirely clear. Nothing was shoved under the hood into library files. Changing any aspect of how they worked was within reach. They could be written in any language, by any developer. And on the client side, we could do anything from compiling as static to using something like Backbone to make a single page app (or both). Technology moves fast, and Rails was too big and heavy of a project too keep up. For an agency like us who deals with so many different types of projects at such a fast pace, we needed the flexibility, and when we started to taste it, there was no turning back.
At the same time, other people started finding the need for flexibility as well. Being able to make single page apps became more important as web apps gained prominence over web sites, and Rails was terrible at this. Mobile apps on multiple platforms started to become essential, and shared data between a web and mobile app became a necessity, which APIs are fantastic at, but again Rails had trouble handling because of their strong server/client coupling. Rails was monolithic and slow compared to a light api/client pairing, and was more expensive to host. And Rails continued to fall off.
And as it did, we gave up consistency for speed and flexibility.
The Mirror on the Front End
Let’s look at the state of front-end development today. We started with normal css, then css preprocessors became the big thing. While there are a few viable preprocessors, SASS generally is perceived as the top of the line. Sass is a great preprocessor, no doubt, and provides a similar benefit to that of Rails — consistency. It has a large, fixed spec that can’t be changed or modified. This is how you make and use variables. This is how you nest. This is how you do a for loop. You get all of it, together, and can use what you want, but you cannot pick and choose, or change the syntax unless you want to fork the project. It’s convenient — you load it up and there’s everything you need, and everyone else who uses it gets the same exact things. And this is the same for other large preprocessors like less and stylus.
But we’re starting to see a change in front-end tooling precisely the same manner that rails started to die out. As soon as postcss entered the game, people started to see the benefits and it rapidly began gaining popularity. Postcss is not so much a preprocessor as a css transformation platform. It doesn’t actually do anything on its own other than parsing css — you have to feed it small, modular plugins that make certain transforms. You can feed it a plugin that adds all the same syntax as sass or less if you want. Or you can feed it one that just adds the capability to use variables, in a different syntax. Or one that adds some extra color modification functions. Whatever you choose. And because you can choose just what you want, it’s really fast. You give up the consistency of a large fixed spec for speed and flexibility.
But why stop at CSS? Babel entered the game as a way to transpile ES6 to ES5, but with the release of v6, converted themselves to the javascript equivalent of postcss — a platform upon which you can load plugins that transform javascript. Out of the box, Babel does no transformation at all. Then you choose your stack of plugins and it starts working its magic. And as soon as people started feeling the power of this, other fixed-spec javascript compilers like coffeescript started to fall off.
I bet you can guess what’s coming last — the final holdout, html. Long story short, there is posthtml, which does all the same things, and is going to be equally disruptive to template engines like jade, slim, handlebars, etc. So now every part of your front end stack can be built from plugins on top of a foundation that offers a parser and a plugin API (and as a matter of fact, this is precisely the principle and stack that spike is built on).
But What About Consistency?
In all of these cases, the web development community has started with complete disorganization, been put into line with large, popular, well established convention-over-configuration frameworks, then slowly moved off of these frameworks to a cleaner, faster, independent-module-based system. And this is good, or else it wouldn’t have happened. There are tons of speed, clarity, and flexibility gains to be had from this type of move. But with every gain comes a loss as well. So what has happened to our consistency?
There were days where a developer could come work from another company that had a Rails site to work with us at Carrot on a rails site without issue, taking everything they learned at the last company and applying it directly to their next project with us. That was a massive advantage that comes with strictly enforced consistency in a popular framework — even between two entirely independent companies consistency could be found. No training time, no ramp-up. No beginner stages.
Today, when new developers come to Carrot, we generally need a week or two to introduce them to the way we do things and get them used to it. Chances are, they have never been anywhere else that has done things like this, and may never again (unless they win the next company over with our obviously superior ways 😉). While the way that we work now is extremely efficient, it is not extremely consistent across the greater community. In fact, hardly anything is these days, just because of the types of tooling that is being adopted. When you build based on plugins, how things are built is purely your choice, and with massive libraries of plugins, everyone’s choices tend to spread out more.
Bringing Back Consistency
I’m not going to lie, I have been wrapped up in working on developing our toolchain here at Carrot for years, and have been so absorbed with creating a workflow that is so efficient and easy to understand that I haven’t zoomed out to look at the greater picture. We are not consistent. We have no single document that declares what sort of tools we use, and how to use them. We don’t have a manual that can be read to understand how to use these tools. But despite that, we have chosen our tools for a very good reason, years of painstaking research, discussions, and testing have gone into the choices. As an agency, we run through projects very quickly, and they range from a microsite with less than 1000 hits a day to a complex web app with millions. We have an unprecedented ability to test out and evaluate different kinds of tools, and we have taken full advantage of that.
But we are so far into our own bubble that for many years we hadn’t taken the time to really sit down and write up the rationale behind our choices, add a detailed manual on how to work with our tools, and shared that with the world. And if we had earlier, I feel like it would not have only potentially helped other freelancers and agencies in similar spots to choose a toolset, but also helped us to get back a little of the consistency that we have lost over the years. We might not be able to find “node developer that like split-client projects and uses spike” in the job listings — someone who is already familiar with our stack and way of working (like we used to be able to hire “rails developers”). But at very least we’d have a clear canon, like the monolithic frameworks of old, that details how and why we do things the way that we do. At least we have done our best towards gaining back that consistency we have slowly leaked.
Now, we do have one, and we’re calling it spike standards. This document describes the features that we look for in the tools that we use to work with HTML, CSS, and Javascript. It is the canon for all developers coming here to work on web-based projects, and the foundation that we provide for all new users of spike. It’s also open to feedback, and will doubtlessly change over time. However, it’s our standard, and we are confident that any changes coming in will be thoroughly reviewed and discussed, to ensure that they have a good balance of efficiency gains and accessibility.
And the best part is that since everything is built on plugin-based architecture, anyone else can take or leave whatever they want with no consequence. Well, other than needing to write up their own canon when it evolves into something different, if they want to be clear to their own developers and/or gain community support. And I think that, over time, while modular architecture is here to stay, there will be certain toolkits and ways of working that become more standardized across the community. And with that, we will come back to a place where developers can rely on strong conventions. But instead of the conventions being cemented into a complex codebase, they will be written documents describing a set of modules that someone prefers and why. And that way, finally, we will be able to get speed, flexibility, and convention.
If you liked this article, click the little green heart below to show your support. Thank you!
