Opting out of monolithic frameworks


In this article I'm going to take an outside look at the lessons learned from following a no-framework approach to developing a web application in a lean team. I’m going to start by looking at the motivations for doing so and then look back at the actual effects and stumbling block that we experienced in a real team.

Churn

With the Javascript world being such a fast moving and vibrant community, Churn is an ever present complication, the deluge of new libraries, changes to existing frameworks and both front and back-end applications makes keeping up no mean feat. One of the notable examples of the rate of change in Javascript as a negative is the recent announcement that Angular JS version 2 will maintain no backward compatibility. The rapid change in existing frameworks coupled with the fact there are just so many option for front-end Javascript applications makes committing to learning huge frameworks seems rather risky.

Committing to a large framework could result in the ability to rapidly develop an application without reinventing the wheel but, in the Javascript world the notion of careful stewardship and long term support seem to me rather lacking. By committing to large framework you’re committing to all of the cognitive overhead of learning its crazy ways. If they pull the rug out from beneath you, the expense of using the framework increases again, you must re-learn it’s new crazy way or risk being left in the doldrums. Maybe there’s another way…

No Frameworks

One proposed solution becoming increasingly popular is to adopt a no-framework approach. A topic of discussion within ThoughtWorks in the past couple of years, no-framework approaches to development allow people to focus on using what is needed when it is needed, something incredibly valuable inside a team following the LEAN startup model.

Keeping Lean

The values of a lean development process indicate the value in releasing early, refining based on user feedback and keeping your application open to change. By adopting a no-framework approach the reduced cognitive overhead for developers should increase early development speed and thus earlier release dates can be achieved. It also promotes an iterative approach of developing features as an MVP using only the bare minimum of tools required. This allows us to align our architecture with the way we split stories. By splitting stories in such a way that we can iterate over the business value we can also iterate over architecture to be more responsive to requirements (Agile Mona Lisa).

Case Study


Timeline

The overarching goal of the application is to pull together the functionality of two external system and provide a way to interact with both of them inside the context of the actual business domain without needing to know which actions involve which external system. The aim of this is to increase user-ship across the two external platforms by making the user experience far better. In the rest of this post I’m going to talk through in roughly chronological order the choices made as the project progressed and the effects of said choices.

Server side framework choices

Everything the system does for you it also does TO you

For rapid development the natural choice for many is to opt in to Rail. The problem with that approach is that we use internal projects as incubators for new developers and we often roll people onto projects with a goal that they should ramp up in a skill they don’t initially have. Give a non ruby coder a rails codebase and suddenly the learning curve is incredibly steep! Another motivation for avoiding the huge monolith of rails can be summed up nicely by Leaver’s Law: “Everything the system does for you it also does to you”. A top chef doesn’t use a swiss army knife when cooking, they have a specific and highly performant tool for each different job! I believe this is the approach we should be taking when developing small applications. Among other factors this caused the team to pass over the idea of using rails.

Sinatra is a micro-framework which removes the complexity of dealing with HTTP directly in ruby but doesn’t overwhelm in the same way of Rails. For smaller applications with only a small number of end points Sinatra seems like the natural choice. Sinatra unlike Rails makes virtually no attempt to solve your problems for you, when developing lean this is a huge win!

The initial layout of the project saw one Sinatra application running to serve data from an external service acting as a facade to the complexity of dealing with our external data source. Another Sinatra application acts to take the data retrieved from a variety of sources and build composite representations of the data and, page templating. In our early use case Sinatra covers this well.

Templating and consuming data

HTML templating could've been the first step towards building our page view however, with Sinatra offering the ability to plug-in a wide range of reputable templating engines the team gained options for providing ruby logic and less verbose templates than bare HTML would've resulted in. The choice of Slim for templating was not the only choice available but offers a very mature solution to HTML templating. One of the clear benefits of this that’s stuck around for the whole project duration is the ability to easily control feature toggles with ruby code.

The initial implementation of our client side API consumption was written using jQuery and Javascript. Rather than choosing to commit to a Javascript framework too early on, the team just implemented what was needed to consume the API and manipulate the DOM accordingly. This was a huge speed advantage in the early days of the application’s development. The UX focus was on providing functionality so people could begin using it and start giving us feedback. By releasing as early as possible we got feedback earlier and began to iterate over features to improve the UX based on real user feedback.

Knockout JS

Early on the speed gains of being close to the metal were vital to an early release. But once we started to improve the user experience it became more and more of a time drain writing DOM manipulation to deal with managing the state of variables in Javascript based on user interaction. As the demands of how we wanted to improve the User Experience increased it because more difficult to manage the DOM manipulation.

We needed an abstraction but we didn't need to reinvent the wheel. We could've began with a Javascript framework and avoided this pain, but we waited until we felt the pain before we assumed we needed a solution for it. Knockout JS provides a great solution to all of this complexity and a neat separation of concerns. Logic lives in the View Models and KO manages our DOM through bindings and observable.

Client side routing

After bring in Knockout we noticed our application was becoming very top heavy. The whole application consisted of a bunch of nested viewmodel instances. We wanted to split these out and give each of them routes. To do so we first had to decide upon a routing framework to manage our client side URLs. After some discussion with people in the Javascript space we were pointed towards director.js a very light Javascript framework for managing client-side routing.

After pulling in the client side routing and writing some framework style code to load views and viewmodels and bind them to a container based on the route. This gave us the very basic functionality that some of the larger frameworks also provide. I think its important to note that we didn't need any of this functionality until 12 month into the development of the project. By bringing it in later we let our application shape the infrastructure and not vice versa.

Grape and Swagger

Following a move towards opening one of our services within the company we decided we would be much more useful if we could generate documentation about or API. We've been looking at swagger for a while and its really cool, by moving over to grape we gain a much more prescriptive model for describing our API endpoints. There are two effects of moving over to grape, firstly we gain more control and a greater robustness to our API, we've gotten lots of validation and model standardization automatically and we have a DSL focused on writing an API.

Lessons

  1. Early development speed gains are great for a openly LEAN model. As long as it’s viable to release a reasonable solution early, and, stakeholders accept that this release will be improved in response to feedback. This means we can prove or disprove our development hypothesis ASAP.
  2. We've found that we can be very responsive to changes in direction. Sure we might need to pull in a new library to enable it but, we have so much less to worry about when doing so. We can change our architecture to fit the business requirements.
  3. After a year we’ve ended up with something that roughly resembles a MVVM framework. It’s got routing, view resolution, view model resolution and a bunch of helpful modules. This is a rough feature parity with existing frameworks.
  4. Looking back at our choice of library for the routing framework we’ve felt a fair amount of pain in connection with that choice. We need features which aren't already included inside director. Director gives you the ability to implement them with regular expressions but, we don't want to be worrying about the regular expression required to match URL parameters when defining a route. Other libraries offer solutions to these patterns and luckily there is a story we can play to pull our director and replace it with a more suitable choice. Would we be able to do this if the routing framework was baked in to a monolithic framework?

External Linkage

  • The State of Javascript in 2015 Jimmy Breck looks at the state of Javascript in 2015, part of his argument is a prediction that people will choose to use a combination of smaller libraries to avoid the pain of churn in the Javascript world.
  • The Agile Mona Lisa provide a good example of the difference between iterative development and incremental development.
  • Leaver’s Law “Everything the system does for you it also does to you.”
  • Knockout JS two way binding JS micro-framework.
  • Sinatra is a DSL for creating web application with minimal effort.
  • Grape is an opinionated micro-framework for creating REST-like APIs in ruby.
  • Swagger is an API documentation framework which gives you the ability to interact with the API through web forms.
  • director.js is an awesome and tiny routing library for Javascript.