More Better Unified JavaScript

Ever since Spike Brehm wrote about isomorphic JavaScript two years ago, I have been somewhat obsessed. I was working on a Backbone app for MESH01 at the time and although I loved client-side JavaScript, I ran into a couple things that were extremely frustrating.

1. Code Duplication — We were using .NET for the back end and there was a lot of the same logic implemented for both .NET and Backbone.
2. SEO — We had a couple publicly facing web pages where SEO was important. For those pages, I ended up implementing the same view logic and templates twice. Once in .NET for the server side rendering that was indexed by search engines. Then again in Backbone for snappy client side rendering once the app was loaded in the user’s browser.
3. Performance — The initial page load was something like 6 seconds. I remember some of our corporate customers claiming it took 10 seconds or more. Even though our app was not open to the public this caused a lot of heartache.

Reading Spike’s article, a light bulb went off in my head.

“…application logic is somewhat arbitrarily split between client and server, or in some cases needs to be duplicated on both sides…You tend to have a Ruby (Python, Java, PHP) library that you’ve been using for awhile, but all of a sudden you have to replicate this logic in a JavaScript library…if we have a JavaScript runtime on the server, we should be able to pull most of this application logic back down to the server in a way that can be shared with the client..your great new product can run on both sides of the wire, serving up real HTML on first pageload, but then kicking off a client-side JavaScript app. In other words, the Holy Grail.”

At the time, I had no idea how to build the Holy Grail. I just knew I had to try. I had The Madness.

AngularJS

Fortunately, I didn’t have to wait long to get my chance. I joined GetHuman in January of 2013 and was tasked with building out some new realtime features. This required adding a new, rich user interface with client-side JavaScript.

One small complication. Any changes I made couldn’t adversely affect the current user experience. The most important of this was that all pages had to load in under 1 second.

The need for both a rich user interface and a super fast initial page load is the perfect use case for isomorphic JavaScript. The only problem was that there weren’t any good isomorphic frameworks. It was early 2013 and the closest choices boiled down to:

• Backbone and Rendr — Spike inspired me, but I wasn’t a fan of Backbone and wanted to use something else.
• Derby — My friend, Tyler Renelle, just ran into a major scalability issue which ruled this out.
• Meteor — At the time, they were focused purely on games and realtime apps. SEO and server rendering wasn’t a big concern for them.

Since there was no perfect framework out there, I decided I would just use a client side framework I liked, AngularJS. I figured I would do for Angular what Spike did for Backbone.

Easier said than done.

Up until recently, Angular was tightly coupled to the DOM. The actual DOM exists in a browser, so if I wanted to render Angular on the server I had two options:

1. Fake the DOM — You can use a fake DOM on the server with a headless browser like JSDom or PhantomJS. The problem is that headless browser rendering took 5 seconds or more in my tests. A common solution for this is to have the headless browser generate an offline cache of static pages. That wouldn’t work for me either since we had thousands of pages and cache times needed to be under one minute.
2. Abstract the DOM — Theoretically, you could go through all the Angular code and replace any references to the DOM with some sort of virtual DOM. The Angular core team did try to do this with 1.2 before dropping it and have more recently implemented a DOM abstraction in the upcoming Angular2, but these were massive efforts. Trying to do this myself would require way more work than I wanted to take on.

I have to admit that as much as I liked Angular on the client, I thought more than once that perhaps Angular was not my best choice for this project.

But…then one day we figured it out.

Vanilla JavaScript

Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away. — Antoine de Saint-Exupéry

I had been thinking about things backwards, upside down and inside out. I was trying to add new complexity (server rendering) to something that was already very complex (Angular). Why not instead start off creating the most simple version of what I was trying to achieve in plain, vanilla JavaScript?

Routing

For example, a simplified version of routing could look like this:

{
    defaultLayout: 'basic',
    routes: [
        {
            urls: ['/'],
            name: 'home',
            layout: 'twoColumn'
        },
        {
            urls:   ['/sitemaps/{type}.xml'],
            name:   'answers.sitemap',
            layout: 'none',
            contentType: 'application/xml'
        }
    ]
}

This one JavaScript object contains all of the key unique information to do routing on EITHER the client or the server. In order to implement “isomorphic” routing off of this, an Angular adapter reads in this routing data and automatically generates routes in the Angular UI Router. On the server side, a Hapi adapter similarly feeds this data into the Hapi router. If one day I wanted to switch from Hapi to Express, I would just create a new Express adapter and work off the same routing data.

Data Definitions

What about the API? This is typically a bugaboo that many isomorphic frameworks try to avoid. How can we represent the API in a declarative format similar to routing? The answer is going to be different for each organization, but this is an example of what I came up with:

{
    name: 'article',
    api: {
        GET: {
            '/articles':        'find'
        },
        PUT: {
            '/articles/{_id}':  'update'
        }
    },
    params: {
        find: {
            required: ['where'],
            optional: ['select', 'sort']
        },
        update: {
            eitheror: ['where', '_id'],
            required: ['data']
        }
    },
    fields: {
        title:  { type: String, required: true },
        bucket: { type: String, 'enum': ['one', 'two'] }
    },
    acl: {
        find: {
            access: ['admin', 'user'],
            select: {
                restricted: {
                    user: ['bucket']
                }
            }
        }
    }
}

Don’t worry about the specific values in this example. They are fungible. The point is that I have a basic data definition which can be used by different adapters for specific frameworks (i.e. Angular, Hapi, Ionic, etc.). Some examples of how I have used this includes:

• Generating API endpoints for the Hapi API server
• Generating API client stubs for the Hapi web server
• Generating API client stubs for Angular
• Performing the actual database back end operations for find and update
• Enforcing security in the API
• Enforcing validation in the API
• Enforcing validation in Angular

UI Components

Let’s extend the same approach used with routing and data above to the user interface. A UI component could look something like this:

{
    // declarative stuff
    defaults: {},
    scope: {},
    validations: {},
    styles: {},

    // basic MVC
    model: function () {},
    view: function () {},
    controller: function () {},

    // other stuff
    serverPreprocessing: function () {},
    eventBusListeners: {},
    uiEventHandlers: {}
}

Again, the specific values are moot. To give you an idea of what can be done, however, here is a list of stuff that I generate from one UI component:

• An Angular 1.x directive
• An Angular UI Router state handler
• An Angular2 component (and thus a web component)
• A Hapi UI component
• A Hapi route handler
• A component for an Ionic mobile app

Server Rendering

With all these pieces in place, I have everything I need for server rendering. At a high level the flow goes something like this:

1. The Hapi router maps a URL request to a handler
2. The handler gets the UI component for the page
3. The UI component model() is used to get the initial data
4. The UI component view() is used to get the page template
5. The data and template are fed into a special adapter called Jangular
6. The HTML returned from Jangular is sent back to the client

More Goodies

In addition to server rendering and more effectively sharing code, I discovered that my approach yielded some unexpected side benefits. One of them is the ability to add my own abstractions and opinions on top of other frameworks. The plain JavaScript objects are completely outside of any framework. The code that actually runs in Angular or Hapi or another framework is generated by adapters. So, it is really easy to add custom rules to the adapters to force developers to follow certain best practices or avoid common mistakes. For example:

• Think two-way data binding in Angular is evil? Add a rule so that your adapter throws an error at build time if a developer adds ng-model to a template.
• Don’t like Angular’s verbose syntax? Add your own sugar or default conventions.
• Worried about the migration from Angular 1.x to 2.0? Set up your adapter so you can output to either format.

This is NOT a framework

I have published several libraries that I use to create declarative components and adapters for different frameworks. The thing is, outside of some of the lower level libraries like Jyt or Jangular, many of the libraries are heavily opinionated and specific to our current needs.

While I don’t expect many people would agree with all of our opinions, I do think that there is a general high level approach for sharing code across all layers of your stack that can be extrapolated from what we have done.

1. First, think unification — How can I boil down what I am doing to its most simple form? Can this be defined declaratively? How can I remove all container-specific dependencies?
2. Then, adapt in a generic way — Given a component definition, how do I write generic code for a target framework/container/environment?
3. Prefer pure functions — What can I pull into isolated utility code that has no state or side effects? The great thing about pure functions and FP in general is that it is naturally isomorphic.
4. Leverage other frameworks — How can I best exploit other frameworks? Don’t boil the ocean. All your code is developed in one common development environment but it runs within the context of other frameworks like Angular, React, Ember, Express, Hapi, etc.

In other words, use the most basic of all good programming techniques: REFACTOR YOUR CODE. I would wager that you understand the concept of refactoring. You just may not be used to doing it across your entire stack.

Final Word

This approach is not for everyone. If you already have a large, complex legacy system in place that doesn’t use JavaScript at all layers, rebuilding your codebase is probably not a wise choice. More importantly, if you are not concerned with initial load time, a single page app may achieve the same goal for you in terms of not duplicating your work.

But if you are building or considering building a mass-market, public-facing web-based app, I highly recommend you consider this approach or one similar. You, too, will feel The Madness, when you realize that your code performs beautifully both on the server and abound, in the hands of your users.