Splitting your app into smaller apps using RabbitMQ

Igor Zhivilo
Sep 14 · 3 min read

After many years of development, we realised our app had become too complex, causing development, testing and debugging to be much harder. We decided to do something about it and the first step needed to solve this problem was splitting our app into smaller apps — starting with extracting the messaging mailer, which is responsible for sending all our messages to clients as a separate app. For this purpose, RabbitMQ was chosen as a broker.

A couple words about RabbitMQ

RabbitMQ is a broker for the AMQP (Advanced Message Queuing Protocol) — Messaging Broker

Reasons for using messaging in your applications

  • Reduce complexity by decoupling and isolating applications
  • Build smaller apps that are easier to develop, debug, test, and scale
  • Build multiple apps that each use the most suitable language or framework versus one big monolithic app
  • Get robustness and reliability through message queue persistence
  • Reduce system sensitivity to downtime

Before we start I suggest you to read this post: Event sourcing on Rails with RabbitMQ. This helped me a lot when I was just starting to learn this technology and explain in details all the things I am not going to talk about for the sake of simplicity. Also, it would be a good idea to read about bunny and sneakers gems, and RabbitMQ official documentation.

To better understand the whole process, I will define a couple of things:

Publisher

  • The first app (publisher) called ‘dashboard’ uses ‘message_publisher’ for delivering messages to the broker via a ‘dashboard.messages’ exchange.
  • The name of the exchange for dashboards is called ‘dashboard.messages,’ which is a direct exchange — for detailed explanations of different types of exchanges, look at https://www.RabbitMQ.com/tutorials/amqp-concepts.html
  • ‘bunny’ gem (https://github.com/ruby-amqp/bunny) was used for publishing messages (Bunny is a popular, easy-to-use, well-maintained Ruby client for RabbitMQ)

creating publisher

# '../lib/publishers/message_publisher.rb'
class Publishers::MessagePublisher
def self.publish(message)
x = channel.direct("dashboard.messages")
x.publish(message.as_json, :persistent => true, :routing_key => '#')
end
def self.channel
@channel ||= connection.create_channel
end

def self.connection
@conn = Bunny.new(APP_CONFIG['rabbitmq.amqp']) # getting configuration from rabbitmq.yml
@conn.start
end
end

RabbitMQ configuration

# '../config/rabbitmq.yml'
rabbitmq:
development:
amqp: "amqp://guest:guest@localhost:5672"
# integration:
# amqp: ...
# production:
# amqp: ...

creating and sending message to broker via publisher

class Message < ActiveRecord::Base
# ...

after_commit publish_message, on: :create

# ...
def publish_message
Publishers::MessagePublisher.publish(self)
end

Consumer

  • Second app(consumer) called ‘mailer’ which consumes messages received from broker and sends them to clients using Action mailer.
  • Name of queue ‘mailer.messages’
  • ‘sneakers’ gem — https://github.com/jondot/sneakers, was used for consuming messages (A fast background processing framework for Ruby and RabbitMQ)

creating worker to consume messages

# '..app/workers/messages_worker.rb'
class MessagesWorker
include Sneakers::Worker
from_queue 'cliche.messages' def work(message)
MyMailer.send_mail(msg).deliver

ack!
end

RabbitMQ configuration

# '..config/rabbitmq.yml'
development:
amqp: "amqp://guest:guest@localhost:5672"
vhost: "/"
# integration:
# amqp: ...
#production:
# amqp: ...

Connecting it all together

rake task to create binding between producer and consumer

namespace :rabbitmq do
desc "Setup routing"
task :setup do
require "bunny"
conn = Bunny.new
conn.start
ch = conn.create_channel # get or create exchange
x = ch.direct('dashboard.messages', :persistent => true)
# get or create queue (note the durable setting)
queue = ch.queue('mailer.messages', :durable => true, :ack => true, :routing_key => '#')
# bind queue to exchange
queue.bind('vcita.messages', :routing_key => '#')
conn.close
end
end

Installing and running it all together

  1. Installation of RabbitMQ — brew install rabbitmq
  2. Run RabbitMQ broker — /usr/local/opt/rabbitmq/sbin/rabbitmq-server
  3. Open dashboard’s directory — cd Projects/dashboard ( in my case ) and run rake task for binding — rake rabbitmq:setup
  4. Open mailers’s directory — cd Projects/mailer ( in my case ) and run sneakers worker — WORKERS=MessagesWorker rake sneakers:run
  5. Check RabbitMQ’s admin is alive — http://localhost:15672, check via admin bindings properly configured between exchange and queue

In coming articles, I will talk about error handling in consumer using sneaker’s failure handlers and what we do in producer in case the connection to the broker is lost for some reason. I hope this helps somebody to better understand how all things are connected and working together, or that I’ve at least provided a starting point.

Link to original post in my blog

Splitting your app with rabbitmq

Igor Zhivilo

Written by

I am Software Engineer who loves to learn and experiment with new technologies.

Splitting your app with rabbitmq

Splitting your app into smaller apps using RabbitMQ

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade