Flatiron Labs
Published in

Flatiron Labs

Introducing Railway IPC

Ruby and Elixir Packages for Inter-App Communication

You Want IPC

IPC, which stands for Inter-Process Communication, is something you want if you can answer “yes” to any of these questions:

  • Does your organization have multiple business domains that your engineering team is tasked with supporting?
  • Is it difficult to coordinate engineering work across these domains?
  • Do you find it hard to innovate new tools and platforms without the fear of breaking existing functionality?

The Problem

As our business grows, we need to meet the needs of more and more groups of stakeholders. We need to move fast *without* breaking things. It can feel like there are only two solutions:

  • Microservice all the things! And expend a ton of time and effort figuring out data syncing requirements, event tracking, cross-app authentication and authorization and more.

The Solution

Adopting asynchronous messaging patterns between applications gives autonomy between teams and encourages us to build small. It allows us to communicate facts and build a set of applications that can run independently of one another. It produces a system that is highly cohesive and lightly coupled.

Railway IPC

  • Synchronous messaging via remote procedure calls (RPC)
  • Part II: How to Use Railway Packages for RPC Calls
  • Part III: Railway Ruby Gem Under the Hood
  • Part IV: Railway Hex Package Under the Hood

Part I: Async Messaging with Railway IPC

In the Railway async messaging pattern, consumers bind a RabbitMQ queue to an exchange and listen to messages on that exchange, over the queue. Publishers publish messages to an exchange.

Learn.co Cohort Creation

At The Flatiron School, we need to create “cohorts” into which we can enroll students. Let’s say we have three apps:

  • Learn.co: A Rails app and our Learning Management System through which students access lessons
  • Registrar: An Elixir app for admitting students and billing them
  • Creating a “registration cohort” in our Registrar app, so that we can admit students into a cohort for a course and bill them their tuition.

Defining a Publisher in Elixir

Having included the Railway package in your app, simply define a publisher module that specifies the exchange to which it will publish.

Defining a Consumer in Ruby

We’ll need to write a consumer class that:

  • Specifies which messages it cares about consuming
  • Specify the handler with which to process those messages

Specifying the Queue and Exchange

We define a consumer, inherit it from RailwayIpc::Consumer, and specify the queue and exchange with the listen_to class method.

Specify the Expected Message and Message Handler

Then, we define the type of message and handler with the handle class method.

Implementing The Handler

We define a class that inherits from RailwayIpc::Handler and implement a handle method that takes a block.

Starting the Consumer

The Railway IPC Ruby Gem provides a handy rake task we can use to start up consumer:

Defining a Publisher in Ruby

Just like when we defined our Elixir publisher, we only need to give our publisher the exchange to which it will publish. We define a class that inherits from RailwayIpc::Publisher, and specify the exchange:

Defining a Consumer in Elixir

Just like when we defined our Ruby consumer, we need a consumer that:

  • Specifies which messages it cares about consuming
  • Specify the handler with which to process those messages

Specifying the Queue and Exchange

We define a module that uses the RailwayIpc.EventsConsumer behaviour, and specifies the exchange to which it will listen to and the queue to bind to that exchange:

Implement the Handler Function for the Expected Message

We need our consumer to implement a handle_in/1 function that uses function arity pattern matching to handle the expected message, in this case the ”BatchCreated” message. It will handle this message by doing some work and then publishing the ”CohortCreatedMessage”.

Starting the Consumer

We want our consumer to start up when the app starts up, and we want our consumer to be supervised so that it can restart in the event of a crash. We’ll add the following to our application.ex file, in the start/1 function:

Message Contracts with Google Protobuf

How can we ensure that consistent messages are passed between applications and that our packages can reliably decode and operate on them? We’re leveraging Google Protobuf to establish shared message contracts across applications.

  • Implementing our Ipc::Handlers::CreateBatchHandler to operate on the data within this message.

Conclusion

The Railway libraries allow us build out robust async messaging systems in Elixir and Ruby with very little hand-rolled code. They offer sane and easy-to-use APIs for defining consumers and publishers in both languages, and, together with Google Protobuf, help our team enforce consistent message contracts between applications. All in all, we hope that these tools will empower us to meet the complex needs of our growing business, allowing relatively autonomous teams of engineers to grow and change our technology with minimal breakage.