Dissecting Rails monolith. Basics.

How and why one may refactor rails app into pack of rails engines

TL;DR: Why? Reveal dependencies. Make technical debt manageable. Save your cognitive budget by having smaller context. How? Extract stylesheets, then migrations and models as core engine, put controller namespaces into separate engines, and move common controller stuff to the core. Then extract gems/engines as you go.

Intro

While Lines Of Code(LOC) is quite controversial measure, most controversy is about using it for productivity measurement, it is not that bad for other things, so I’ll go with it.

I’ve seen few legacy apps out there — and every app becomes legacy app sooner or later. Sure every app is different in many aspects: used tools and versions of those, presence and depth of tests, app/lib balance — some apps are like icebergs: simple web UI and tons of backend machinery to interact with multiple 3rd party services.

But almost always, somewhere in between 10000-15000 LOC things start getting messy.

Technical debt

If you are lucky/good then technical debt, that you have at this point (10-15 KLOC), is manageable, if not — you have to stop active development, invest resources into refactoring or default on technical debt (stop or kill the project).

Technical debt itself is big topic, so let’s go to conclusions right away:

Technical debt is always there — no matter how good you are

Size of technical debt is not actually known — this is why it is hard to manage

Splitting your app into somewhat isolated parts allows you to see big problems before they become very big problems — simple risk management

Dependencies and modularity

Software modularity is another important topic, and this topic is really opinionated, I guess few books are written about it, but we don’t need to dive that deep.

Regardless of your opinion on how good modularity is, and how one should achieve it, one thing is certain:

Explicit, visible dependency is much better than implicit, hidden one. Revealing dependencies is a first step for managing them in any way.

Namespaces won’t save you

Controller namespaces are good start. We all had ‘/admin’ namespace in our apps, right? In fact somewhere in between 1000-2000 LOC I was pretty confident that controller namespaces is all I need to keep parts of my app isolated. But after reaching 2000 LOC I’ve noticed that controller namespace don’t affect business logic layer, so I’ve decided give engines a try.

Choice criteria

I guess you have your opinion about how good is separation/modularity of your app. You see, once your app is split, merging it back into single piece is no-brainer, right? Well, let me challenge you:

If you may split your 5000+ LOC app into engines within 1 day and be happy about it, then I guess you are OK already, you got your modularity somehow (and you should write a post or two about it, really)

I failed to meet this criteria, and this post is for those who failed it too

Failure scenario — database is big

I had 3 controller namespaces already, and expected fast app split, well, surprise — database is bigger dependency than one might think. Ben Smith covered that for us in his bright post about how to extract migrations into engines, when you have multiple engines in development

Failure scenario — split may go wrong

I’ve seen “app” that was split into 8(!) “services” with enormous code duplication in front-end and backend. One should split and merge wisely to avoid such case. Technical debt is in many ways a collection of wrong decisions, and once technical debt reached critical mass, it becomes obvious that it will never be repaid.

Success scenario

Our goal is to split original app into pieces, as a result of that split, initial app, will be almost nothing, but a container for our engines:

This is our final Gemfile:

source ‘https://rubygems.org'
# For now just use local gems with engines in ./gems directory
gem ‘project_style’, path: ‘gems/project_style’
gem ‘project_core’, path: ‘gems/project_core’
gem ‘project_store’, path: ‘gems/project_store’
gem ‘project_theme_store’, path: ‘gems/project_theme_store’

And this is our config/routes.rb file:

Rails.application.routes.draw do
  mount ProjectStyle::Engine => “/style”
  constraints(ThemeStoreSubdomainConstraint) do
    mount ProjectThemeStore::Engine => “/”
  end
  constraints(StoreSubdomainConstraint) do
    mount ProjectStore::Engine => “/”
  end
 
end

1. extract project_style engine for all your CSS/SASS/LESS, and style images
2. extract project_core engine with all migrations and model classes and model tests — read this guys, about how to do it http://pivotallabs.com/leave-your-migrations-in-your-rails-engines/
3. be sure that all shared 3rd party gems are listed at gemspec dependencies of project_core (carrierwave, caminari, devise, inherited_resources, etc)
4. extract common controller concerns to the project_core
5. extract all controller namespaces as separate engines
6. extracts gems from your lib, and include them in project_core/some_engine

This is it — you’ve just set solid foundation for app de-coupling.

To be continued

As you see, I don’t talk about:

1. using Message Queues or REST for communication between apps
2. dealing with shared tests & factories
3. managing release cycle or repository structure
4. dependency graph of engines, common mistakes

All those topics are interesting and big, and should be covered one by one. Stay tuned.