How I created engines from monolithic Ruby on Rails application!

Photo by Samuel Fyfe on Unsplash

In this post, I will be sharing my experience of extracting out engines from a monolithic Ruby on Rails application. In my application, there were several modules written to function together establishing association with each other. I recently had a business requirement where the application should be flexible to install in client’s on premise platform and each client might need some of the modules (Not All). This was the time when my team realized that we need to extract some of the additional modules to independent unit, so that we can easily lean the monolithic application to keep only the bare minimum modules and add the additional modules as needed.

Since our objective was clear to create independent modules, the obvious choice for us was the Rails Engine which will help us to create miniature Rails like application and they can be easily installed/uninstalled from the application.

I assume you have gone through the Rails guide for engine and understand the way engine functions.

Preparing main_app

Since we will be extracting out some features, it would be easy to manage if we can create a separate git repository for it. First of all, I added an entry /engines/* to .gitignore file of my monolithic application (further I will refer it as main_app) and then created a directory within root path of main_app called engines (mkdir engines) .

Create Engine

Now its the show time to target a module and create a Rails engine for it. First of all, I went inside engines directory and created a mountable Rails Engine using command:

rails plugin new my_engine --mountable
# edit my_engine.gemspec to enter required homepage, summary, description details

Extracting Gems

The module which I was extracting, had some gem dependencies which had to be moved to my_engine.gemspec. For an example the target module was sending text messages to users using Twilio service. So I moved the twilio-ruby gem from main_app Gemfile to engines/my_engine/my_engine.gemspec adding the following line:

s.add_dependency “twilio-ruby”

And added my_engine as a gem to main_app’s Gemfile, so that we can load the engine and bundle its dependencies:

gem ‘my_engine’, path: ‘engines/my_engine’

Extracting Configurations

As I extracted the twilio-ruby gem to my_engine, now It made sense to move the twilio initializer file (main_app/config/initializers/twilio_key.yml) to my_engine (engines/my_engine/config/initializers/twilio_key.yml) path.

I have configured the ACCOUNT_SID, AUTH_TOKEN, FROM etc. in twilio_key.yml which is required by twilio-ruby gem to validate my Twilio account to send text messages.

Similarly, all other dependent configuration files should be moved to its own engine.

Extracting Routes

I went through the main app routes file and copied the relevant routes for engine to engines/my_engine/config/routes.rb file.

Extracting Controllers

Then I identified the engine controller classes from routes and moved them to engines/my_engine/app/controllers/my_engine/ path. However, my controllers still use some methods of main_app’s ApplicationController .

So, I decided to inherit main_app’s ApplicationController to my controller classes:

module MyEngine
 class ExamplesController < ::ApplicationController
 end
end

Similarly I moved engine related assets, helpers, mailers, workers, services, etc to my_engine from main_app.

Extracting Views

Like controllers, I also got the related views folders from main_app and moved them to engines/my_engine/app/views/my_engine/ path.

In my application, the main_app had links that points to my_engines views. So, I had to update the links (*_path) within main_app to refer the routes of my engine by writing my_engine.*_path and and similarly I updated the engine views’ links that refer to main_app’s routes with main_app.*_path .

Extracting Models

I identified the core models of the engine from main_app and analyzed the associations (has_many, belongs_to, etc) to get a fair idea which were parent models and child models of each model. Then I moved all the relevant model files from main_app to engines/my_engine/app/models/my_engine/ .

Now, the challenge was to rewrite associations in my_engine models to establish connection between my_engine model and main_app model.

I achieved this connection using the following steps:

1. Since my motto was always to install and remove the engine easily from main_app. I thought of creating an initializer file in main_app and passing main_app model names to engine. So that engine can smoothly establish the connection.

I created the initializer file main_app/config/initializers/my_engine.rb with following content:

MyEngine.configure do |config|
 config.example_klass = “Example”
end

In my application, I had to pass multiple model names to my_engine. So I thought to automate generation of this preset initializer file with the help of Rails generators. I created one template file for my generator which will exactly look like the initializer file in fact I named it as engines/my_engine/lib/generators/templates/initializer.rb with the exact same content as the initializer file we have seen above already:

MyEngine.configure do |config|
 config.example_klass = “Example”
end

Then I created the following generator script in the path engines/my_engine/lib/generators/my_engine/install_generator.rb which will just copy initializer.rb to main_app/config/initializers/my_engine.rb path.

module MyEngine
  class InstallGenerator < Rails::Generators::Base
    source_root File.expand_path("../../templates", __FILE__)
    desc "Configure necessary files to use MyEngine"
    def copy_initializer_file
      copy_file "initializer.rb",
                 Rails.root.join("config/initializers/my_engine.rb")
    end
  end
end

So, whenever I install my_engine I just need to run one single command to generate this initializer file: rails g my_engine:install

2. Then I created a file engines/my_engine/lib/my_engine/configuration.rb and it looked like below. Basically I defined configuration attributes which will be used to store the main_app model names in the above initializer file.

module MyEngine
  class Configuration
    attr_accessor :example_klass
  end
end

3. Now, it was the time to create the configure hook and to define a getter method using value of configuration attributes (example_klass) to return a constantize class name (Example) of main_app model.

So I updated the engines/my_engine/lib/my_engine.rb file and it looked like below:

require "my_engine/engine"
require "my_engine/configuration"
require "twilio-ruby"
module MyEngine
 class << self
  attr_reader :config
  def configure
   @config = MyEngine::Configuration.new
   yield config
   define_config_getter_methods!
  end
  def define_config_getter_methods!
   config.instance_variables.each do |attr|
    define_singleton_method attr[1..-1] do
     config.public_send(attr[1..-1]).constantize
    end
   end
  end
 end
end

The purpose of writing the define_config_getter_methods! method was just to be able to write MyEngine.example_klass anywhere within engine to return Example class of main_app.

Now, its very convenient for my_engine models (example: IncidentReport) to establish association with main_app’s Example model just by writing below:

belongs_to :example, class_name: MyEngine.example_klass.to_s, foreign_key: :example_id

But in my main_app’s Example model, I had a has_many association with my_engine’s model class (IncidentReport). I did not want to hard code these associations in main_app model class as these were related to my_engine only. So if I had hard-coded it, then each time I had to write it while installing the my_engine in main_app and also had to remove it while uninstalling the my_engine. So I decided to patch these main_app classes from my_engine itself.

So I created a patch file engines/my_engine/app/models/my_engine/concerns/acts_as_example.rb to add the association in main_app’s Example class during runtime and its content looked like below:

module MyEngine::Concerns::ActsAsExample
 extend ActiveSupport::Concern
 included do
  has_many :incident_reports, 
            class_name: “MyEngine::IncidentReport”
 end
end

And then specified in my engine to include my patch during runtime:

module MyEngine
 class Engine < ::Rails::Engine
  isolate_namespace MyEngine
  config.to_prepare do
   MyEngine.example_klass.send(
    :include, MyEngine::Concerns::ActsAsExample)
  end
 end
end

Reference: Thanks to this article for sharing the Engine patterns.

Extracting Migration

Since I had moved all the models and they are name spaced (engines/my_engine/app/models/my_engine) within my_engine, then I thought of maintaining the name spacing in table names as well.

Prior to creating this engine my table name was incident_reports, after engine it became my_engine_incident_reports .

Then I referred the main_app’s db/schema.rb and recreated a migration file within my_engine to create an exact structure for the models which were extracted to my_engine.

Once the migration file was ready, from main_app’s root path I ran the engine task rails my_engine:install:migrations to copy that migration file to main_app’s db/migrate path and followed by this I ran rails db:migrate to run this added migration file.

If you need to backup old data and restore, make sure to do that before removing unnecessary old engine related migration files.

Done!

Finally I tracked down the migration files which were created initially to create the engine related schema. I safely ran those migrations down with VERSION number and gradually removed them from source code.

With this setup, I was able to launch the application with my_engine functionalities.

Thank you for reading through the post and please do comment to let me know if I can improve this setup.