Building Micro-services using gRPC on Ruby

Today, REST with JSON is the most popular framework amongst web developers for network communication. But, it is not very suitable for a micro-services architecture mainly because of latency added by JSON data transmission / serializing / deserializing.

My quest for finding an optimal network communication framework for micro-services brought me to gRPC.
“gRPC is a modern, open source remote procedure call (RPC) framework that can run anywhere. It enables client and server applications to communicate transparently, and makes it easier to build connected systems.”

To read more about benefits of gRPC, visit the official site here.

Serialization in gRPC is based on Protocol Buffers, a language and platform independent serialization mechanism for structured data.
“Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data — think XML, but smaller, faster, and simpler.”

In the remaining part of this post, I will be walking you through setting up a simple gRPC server from scratch on Ruby. Let’s build Snip — a dummy URL shortener!

We will divide our code structure into 3 separate repositories:

1. snip : contains the proto definitions and converted ruby files for client communication. Basically, this is like an interface between client and server, specifying the RPC methods, and the request and response formats.
2. snip-service : Service implementation for the RPC methods (This is where the gRPC server sits).
3. X-app : This is any application who wishes to call snip-service for shortening URLs.

snip will be packaged as a gem, and included in both snip-service and X-app.

Step 0: Install dependencies

Make sure you have ruby and bundler setup working. Then, install the required gems for grpc:

gem install grpc
gem install grpc-tools

PART A: snip gem

Step 1: Setup snip gem

snip is supposed to be a ruby gem, so you could use the bundler scaffold for creating it.

bundle gem snip

Add this to snip.gemspec file:

spec.add_dependency "grpc"

Step 2: Define proto files

Let’s create a new file proto/snip.proto

syntax = "proto3";
package snip;
service UrlSnipService {
    rpc snip_it(SnipRequest) returns (SnipResponse) {}
}
message SnipRequest {
    string url = 1;
}
message SnipResponse {
    string url = 1;
}

Step 3: Generate ruby bindings for the proto definition

Next, we are going to convert the defined proto files to ruby bindings, which are ultimately going to be used by both the client and the server.

grpc_tools_ruby_protoc -Iproto --ruby_out=lib --grpc_out=lib proto/snip.proto

My snip directory tree after this command:

├── Gemfile
├── Gemfile.lock
├── LICENSE
├── README.md
├── lib
│   ├── proto
│   │   ├── snip_pb.rb
│   │   └── snip_services_pb.rb
│   └── snip
│       └── version.rb
├── proto
│   └── snip.proto
└── snip.gemspec

Note that snip_pb.rb and snip-services_pb.rb are the important files which are required for client-server communication. This is how they should look:

snip_pb.rb

# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: snip.proto
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
  add_message "snip.SnipRequest" do
    optional :url, :string, 1
  end
  add_message "snip.SnipResponse" do
    optional :url, :string, 1
  end
end
module Snip
  SnipRequest = Google::Protobuf::DescriptorPool.generated_pool.lookup("snip.SnipRequest").msgclass
  SnipResponse = Google::Protobuf::DescriptorPool.generated_pool.lookup("snip.SnipResponse").msgclass
end

snip_services_pb.rb

# Generated by the protocol buffer compiler.  DO NOT EDIT!
# Source: snip.proto for package 'snip'
require 'grpc'
require 'snip_pb'
module Snip
  module UrlSnipService
    class Service
      include GRPC::GenericService
      self.marshal_class_method = :encode
      self.unmarshal_class_method = :decode
      self.service_name = 'snip.UrlSnipService'
      rpc :snip_it, SnipRequest, SnipResponse
    end
    Stub = Service.rpc_stub_class
  end
end

Voila! You are done with your snip gem. We will move onto the service implementation next.

PART B: snip-service & gRPC server

Step 1: Setup

Add the following to your Gemfile in snip-service

gem 'snip',:git => "https://github.com/shiladitya-bits/snip",:branch => 'master'
gem 'grpc', '~> 1.0'

Replace snip gem path with wherever you have setup the gem to be in.

Step 2: Service implementation

lib/services/snip_service.rb

require 'grpc'
require 'snip_services_pb'
class SnipService < Snip::UrlSnipService::Service
  def snip_it(snip_req, _unused_call)
    puts "Received URL snip request for #{snip_req.url}"
    Snip::SnipResponse.new(url: snip_req.url)
  end
end

snip_it is the RPC method we defined in our proto. Let us look at the 2 parameters here:

• snip_req - the request proto object sent by client in the format as defined in proto Snip::SnipRequest
• _unused_call - this contains other metadata sent by client. We will talk about how to send metadata in another post later.

You need to return an object of Snip::SnipResponse from this method as your response. For keeping the implementation simple, we are sending back the same URL as sent by the client.

Step 3: Setup your gRPC server

Now that your service implementation is ready, let us setup the gRPC server that will be serving calls to your service.

lib/start_server.rb

#!/usr/bin/env ruby
require 'rubygems'
require 'snip_services_pb'
require_relative 'services/snip_service'
class SnipServer
  class << self
    def start
      start_grpc_server
    end
    private
    def start_grpc_server
      @server = GRPC::RpcServer.new
      @server.add_http2_port("0.0.0.0:50052", :this_port_is_insecure)
      @server.handle(SnipService)
      @server.run_till_terminated
    end
  end
end
SnipServer.start

A very simple ruby class which starts the server on port 50052 on running this:

bundle exec lib/start_server.rb

You might need to do a chmod +x lib/start_server.rb for giving executable permissions.

PART C: X-app client

Step 1: Setup

Same as the snip-service Gemfile, you need to include snip gem in your client as well.

gem 'snip',:git => "https://github.com/shiladitya-bits/snip",:branch => 'master'
 gem 'grpc', '~> 1.0'

Step 2: Last step: RPC call!

Just a final piece of code which helps you test out your gRPC server:

test/test_snip_service

#!/usr/bin/env ruby
require 'grpc'
require 'snip_services_pb'
def test_single_call
  stub = Snip::UrlSnipService::Stub.new('0.0.0.0:50052', :this_channel_is_insecure)
  req = Snip::SnipRequest.new(url: 'http://shiladitya-bits.github.io')
  resp_obj = stub.snip_it(req)
  puts "Snipped URL: #{resp_obj.url}"
end
test_single_call

There you go! Your first working gRPC communication is complete!

The snip and snip-service repositories are available on Github. You can find a sample client call inside snip-service itself in test/test_snip_service.

This is the end of this post. Check out my latest post on Dockerizing your gRPC service

Find the original post on my blog