Why spirit

Recently I announced a working pre-release of my project spirit. It’s a modern functional approach to building web applications and larger frameworks.

The project thus far took me a solid 2 months to complete up to this point. And normally I wouldn’t commit that much time into working on a project that there was alternatives to, but there wasn’t any.

Having been trying out Clojure and Ring previously, I saw a beautiful development environment (and tools) for web programming with well done abstractions through out. But I started working on a web project for node.js, and thus started using the popular Express library, which I have used awhile back. It became almost frustrating immediately, it was like taking a step back in the development process. The fault isn’t completely with Express, and when it was, I can’t say it was a fundamental fault.

It’s just that the times have changed, and Express never changed

It became cumbersome to have to test my routes, since I’d have to mock a req or res object and attach some listener to know when the res object was written to. There was no easy way to simply call an existing route either because of how embedded it was in HTTP related code.

So you would begin refactoring your code, and in the end you ended up with more code than you really needed. You had to separate the route functions from the actual logic of the function in order to re-use that code logic somewhere else. That basically makes your code foot print double if you had to do that for every route. It makes your routes proprietary by nature, or tied to the node HTTP implementation.

The landscape of javascript is also moving towards being isomorphic when possible. And when you look at some of these isomorphic projects for Express, you see the baggage of unused Express or http code, you see the hacks and work arounds to `req` and `res`.

The issue extends beyond Express routes

It also extends into the middleware too. Since middleware in Express (or Connect) depend on a req and res as well. The other issue with the middleware implementation is they only go one way.

This is a stark contrast to the way HTTP works, like a function, you have input and output:

-> request comes in ( input ) -> web app

<- response goes out (output) <- web app

In Connect / Express, you have the input, but there’s no real unwinding back on the output. It abruptly stops when something somewhere writes to `res`. And the Express source code have checks all over to see if the `res` was written to. And since there’s no unwinding or flow back on the response, Express uses a hack of wrapping `res.end` to inject final response changes.

Implementation also matters

It’s no secret when you look at the source code of Express (and other libraries) that they take a deep dive object approach to implementing their design. The problem is not with objects, but the manipulation of objects and their prototypes that are a cause for concern.

This is an anti-pattern in my opinion. Because it makes Express hard to extend (and even maintain) without also falling down the same road of manipulating objects, their inheritances and prototypes.

Also it is slow. A barebones Express app will run about 2x slower than just running nodejs http standalone. That is a lot of overhead.

The problem is not specific to Express

It’s not that I have an issue with Express, I liked it very much back when it was first released years ago. I also have much respect to TJ (the creator of Express) and his other work, and also learned quite a bit just reading his source code.

The issues I outlined are in some form prevalent in Koa, Hapi, and projects based on Express, and others as well.

But I think there is a cultural issue as well. In the time that has passed, the issues are in some ways embraced. Instead of actual fixes and solutions, the issues have just been worked around, worked around, and worked around to the point that it’s just accepted.

spirit as a solution

spirit is the result of me trying to fix all the issues I mentioned above.

`req` and `res` are abstracted away at the point of entry via an adapter. Which totals really to only 5 lines of code. But you reap the benefits of it through out all of spirit. And as a result routes and middlewares are all just normal javascript functions that return something.

This makes them highly re-usable and testable, and isomorphic in most circumstances.

Middlewares flow both ways in a bi-directional manner. Meaning you can have middleware for the output (response) too, not just for the incoming request. This happens because middlewares are just normal functions that return something, and when it unwind backwards, it can actually act on the return values.

And since routes and middlewares are just normal functions, spirit itself is a mostly functional implementation (as opposed to a more object approach). This allows spirit to be easily extended, even it’s API, as they are just exported functions that can be wrapped or called individually.

This more functional approach also allows spirit to achieve lower latency and higher throughput on requests compared to Express and Koa. A barebones spirit app stays on parity with a simple node.js http app (meaning it is 2x faster than Express and Koa on the up side, and on on the low side, it still out performs them by 50%).

Also to be even more modular and flexible. spirit sports adapters (which I mentioned provides the abstraction for `req` and `res`) which can be written to adapt to other situations. For instance, a spirit adapter for the browser that hooks into DOM events, or if node.js has another web server written for it with a radically different interface, an adapter can be written for that. And all other middlewares and handlers are expected to continue to work. This keeps it forward friendly and easier to maintain and extend.

A different side to things

An anecdote…

When I first showed spirit to friends who are not familiar with node.js and it’s history. They kind of just shrug and said “well isn’t that normal? isn’t that how it’s suppose to be”?

And when I showed spirit to someone who is familiar with node.js but doesn’t spend much time in a functional language, (I assume) they shrugged because they don’t see the benefits of writing applications this way.

But whatever side you are on, or in between or outside of, I hope you will take joy in using spirit. Or at the very least found this article insightful in seeing another side of things.