Bridging Systems with gRPC, A Powerhouse Protocol

In February 2015, Google open-sourced a robust framework for universal Remote Procedure Calls (RPC). This language-agnostic and highly efficient framework revolutionized the way microservices or any distributed systems interact and communicate with one another.

With the gRPC framework, clients can easily invoke methods on servers located on different machines, as if they were local object methods. This makes for a much more streamlined and seamless process, eliminating the messiness that often comes with distributed system communication.

It operates based on the concept of defining a service, which outlines the available methods along with their parameters and return types inside a .proto file. The server implements these methods and operates a gRPC server to handle client calls, while on the client-side, there is a stub or client that offers the same methods as that of the server.

gRPC utilizes the HTTP/2 protocol for transmitting binary data and Protocol Buffers (protobuf) as its interface definition language (IDL). This empowers developers to specify the structure of the service and messages in a particular format, and subsequently generate language-specific bindings for their server and client based on this predefined format.

Protocol Buffers, or protobuf, is a powerful binary serialization toolset and language used by gRPC instead of text-based formats like JSON or XML. Using this, gRPC achieves faster serialization/deserialization of data and reduces payload size. Additionally, gRPC is flexible and developers can plug in custom codecs to support other data formats.

There are several other benefits of adopting this framework, including but not limited to significant advantages over traditional HTTP/1.1 in terms of concurrency, bi-directional streaming, server push, header compression, and more. It is also very developer-friendly due to its API-oriented approach and automatic code generation feature for multiple languages using the protocol compiler.

To get the protobuf compiler, protoc, using a package manager for Linux or macOS, we can use the following commands:

# linux, using apt or apt-get
apt install -y protobuf-compiler

# macOS, using Homebrew
brew install protobuf

To install it on a windows system, follow this awesome article: https://www.geeksforgeeks.org/how-to-install-protocol-buffers-on-windows/

To verify if the compiler installation was successful and to check the version, we can run the following command:

protoc --version
# output: libprotoc 23.3 (should be 3+)

We would also require to install some plugins for the protoc compiler:

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

• protoc-gen-go: Generates Go code for Protocol Buffers.
• protoc-gen-go-grpc: Generates Go gRPC service code.

Note: Make sure the PATH environment variable includes the bin directory of the GOPATH, this will be necessary to run the Go binaries. We can set the path using the following commands:

# unix/linux based systems
export PATH="${PATH}:$(go env GOPATH)/bin";

# windows based systems
set PATH=%PATH%;%GOPATH%\bin

Let’s explore a simple example to understand how to implement a gRPC-based chat service. The project consists of two components: an application written in Golang that can either work as a server or a client and another client app in Node.js.

Our project structure will look like this:

.
├── proto
│   └── chat.proto
├── gen
│   └── chat
│       ├── chat.pb.go // generated by protoc
│       └── chat_grpc.pb.go // generated by protoc
├── server
│   └── server.go
├── client
│   └── client.go
├── main.go
├── js-client
│   └── client.js

Looking into the project structure, we see the following files:

chat.proto

The Protocol Buffers file, commonly referred to as the proto file, defines the data structures and services in a format that can be understood by both the server and client.

syntax = "proto3";

package main;

option go_package = "./chat";

service ChatApi {
    rpc Connect (User) returns (stream Message);
    rpc Broadcast (Message) returns (Message);
}

message User {
    string id = 1;
}

message Message {
    User user = 1;
    string content = 2;  
}

In this file we have defined a ChatApi service with two RPC methods:

1. Connect: This function establishes a connection with a user and starts streaming messages. It takes a User object and returns a stream of Messages.
2. Broadcast: This function takes in a Message and sends it to all the connected clients. It returns a Message object as well.

Additionally, there are two message types:

1. User: It only contains the user's id.
2. Message: It contains a User and the content of the message.

Our next step is to use the protoc compiler to generate the Go code from the .proto files which can be used by our application to interact with gRPC services.

protoc --proto_path=proto proto/*.proto --go_out=gen/ --go-grpc_out=gen/

Let’s break down each part of the command:

1. protoc: This is the Protocol Buffers compiler. It reads .proto files and generates code in the specified language.
2. --proto_path=proto: This option tells protoc where to look for .proto files. Here, it is set to look in the proto directory.
3. proto/*.proto: This specifies the .proto files that protoc should compile. Here, it's using a wildcard (*) to compile all .proto files in the proto directory.
4. --go_out=gen/: This option tells protoc to generate Go code and specifies where to put the generated files. Here, it's set to output to the gen/ directory. The generated files will include message types defined in the .proto files.
5. --go-grpc_out=gen/: This option tells protoc to generate Go gRPC code and specifies where to put the generated files. Here, it's set to output to the gen/ directory. The generated files will include gRPC client and server code for services defined in the .proto files.

After successfully executing the above command, two new files will be generated: chat.pb.go and chat_grpc.pb.go inside a chat folder, serving as the output for options 4 and 5 respectively.

server.go

This file contains the implementation of the gRPC server. It has the StartGrpcServer function which starts the gRPC server, listening on port 8080.

package server

import (
 "context"
 "log"
 "net"
 "sync"
 "github.com/anik-ghosh-au7/grpc-messenger/gen/chat"
 "google.golang.org/grpc"
)

type server struct {
 chat.UnimplementedChatApiServer
 clients map[string]chat.ChatApi_ConnectServer
 mu      sync.Mutex
}

func (s *server) Connect(user *chat.User, stream chat.ChatApi_ConnectServer) error {
 s.mu.Lock()
 s.clients[user.Id] = stream
 s.mu.Unlock()
 log.Println("Client Connected: ", user.Id)
 <-stream.Context().Done()
 s.mu.Lock()
 delete(s.clients, user.Id)
 s.mu.Unlock()
 log.Println("Client Disconnected: ", user.Id)
 return nil
}

func (s *server) Broadcast(ctx context.Context, message *chat.Message) (*chat.Message, error) {
 s.mu.Lock()
 defer s.mu.Unlock()
 for id, clientStream := range s.clients {
  if id != message.User.Id {
   if err := clientStream.Send(message); err != nil {
    log.Println("Error broadcasting message to", id, ":", err)
   }
  }
 }
 return message, nil
}

func StartGrpcServer() error {
 listener, err := net.Listen("tcp", ":8080")
 if err != nil {
  return err
 }
 srv := grpc.NewServer()
 chat.RegisterChatApiServer(srv, &server{
  clients: make(map[string]chat.ChatApi_ConnectServer),
 })
 log.Println("Server started. Listening on port 8080.")
 return srv.Serve(listener)
}

The code above also contains the implementation of ChatApi server, with methods Connect and Broadcast :

• Connect: adds the client’s stream to the server’s clients map using the user’s ID as the key. Then it waits for the client’s context to be done (which means the client has disconnected), after which it removes the client from the clients map.
• Broadcast: sends the received message to all connected clients except the sender. It locks the clients map using a mutex to prevent data races.

client.go

This file contains the implementation of the gRPC client. It has the StartGrpcClient function which:

• Reads the server URL and the client’s ID from the console.
• Establishes a gRPC connection with the server and initiates the Connect RPC with the client’s User object.
• Starts a go-routine that constantly reads messages from the server and prints them on the console.
• Reads messages from the console and calls the Broadcast RPC to send them to the server.

package client

import (
 "bufio"
 "context"
 "fmt"
 "os"
 "strings"
 "github.com/anik-ghosh-au7/grpc-messenger/gen/chat"
 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
)

func StartGrpcClient() error {
 reader := bufio.NewReader(os.Stdin)
 fmt.Print("Enter the server url (example: localhost:8080): ")
 serverURL, _ := reader.ReadString('\n')
 serverURL = strings.TrimSpace(serverURL)
 conn, err := grpc.Dial(serverURL, grpc.WithTransportCredentials(insecure.NewCredentials()))
 if err != nil {
  return err
 }
 defer conn.Close()
 client := chat.NewChatApiClient(conn)
 fmt.Print("Enter your client ID: ")
 clientID, _ := reader.ReadString('\n')
 clientID = strings.TrimSpace(clientID)
 user := &chat.User{Id: clientID}
 stream, err := client.Connect(context.Background(), user)
 if err != nil {
  return err
 }
 go func() {
  for {
   message, err := stream.Recv()
   if err != nil {
    fmt.Println("Disconnected from server.")
    return
   }
   fmt.Println(message.User.Id+": ", message.Content)
  }
 }()
 for {
  messageContent, _ := reader.ReadString('\n')
  msg := &chat.Message{
   User:    user,
   Content: strings.TrimSpace(messageContent),
  }
  client.Broadcast(context.Background(), msg)
 }
}

Note: The code uses insecure transport credentials, which means the communication between the client and server isn’t encrypted.

main.go

This is the entry point for our Go application. It reads from the console and determines whether to start as a server or client:

• If the user enters s, it calls the StartGrpcServer function from server package to start the server.
• If the user enters c ,it calls the StartGrpcClient function from client package to start the client.

If any error occurs during these processes, it logs the error message and exits.

package main

import (
 "bufio"
 "fmt"
 "log"
 "os"
 "strings"
 "github.com/anik-ghosh-au7/grpc-messenger/client"
 "github.com/anik-ghosh-au7/grpc-messenger/server"
)

func main() {
 reader := bufio.NewReader(os.Stdin)
 fmt.Print("Enter 's' to start as a server, or 'c' to start as a client: ")
 option, _ := reader.ReadString('\n')
 option = strings.TrimSpace(option)
 switch option {
 case "s":
  err := server.StartGrpcServer()
  if err != nil {
   log.Fatalf("Failed to start the server: %v", err)
  }
 case "c":
  err := client.StartGrpcClient()
  if err != nil {
   log.Fatalf("Failed to start the client: %v", err)
  }
 default:
  fmt.Println("Invalid option. Exiting.")
 }
}

Now that we are done with our Go app, let’s take a look at the node.js client.

We need to install the following packages inside the js_client folder using npm or any other node package managers.

• @grpc/grpc-js: This package provides a JavaScript implementation of gRPC for Node.js.
• @grpc/proto-loader: This package is a utility for loading .proto files, which define gRPC services.

npm install --save @grpc/grpc-js @grpc/proto-loader

client.js

const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');
const readline = require('readline'); // built-in node module for reading data from a stream

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});

rl.question("Enter the server url (example: localhost:8080): ", (serverURL) => {
  rl.question("Enter your client ID: ", (clientID) => {
    const packageDefinition = protoLoader.loadSync('../proto/chat.proto', {
      keepCase: true, // original field names will be used
      longs: String, // represent long values as strings
      enums: String, // represent enum values as strings
      defaults: true, // set fields to default value if not set
      oneofs: true // generates a property for oneof fields in the message to indicate which field is set
    });
    const chat = grpc.loadPackageDefinition(packageDefinition);
    const client = new chat.main.ChatApi(
      serverURL,
      grpc.credentials.createInsecure()
    );
    const stream = client.Connect({ id: clientID });
    stream.on('data', (message) => {
      console.log(`${message.user.id}: ${message.content}`);
    });
    stream.on('error', (error) => {
      console.log('Disconnected from server due to error:', error.message);
      process.exit(1);
    });
    stream.on('end', () => {
      console.log('Disconnected from server.');
      process.exit();
    });
    rl.on('line', (line) => {
      client.Broadcast(
        { user: { id: clientID }, content: line.trim() },
        (err) => {
          if (err) {
            console.log('Error broadcasting message:', err);
          }
        }
      );
    });
  });
});

This script contains the implementation of the gRPC chat client in Node.js

Here’s a high-level overview of what it does:

• Uses the readline module to create an interface for input and output in the console.
• Prompts the user to enter a server URL and a client ID via the command line interface.
• The chat.proto file, which defines the chat service, is loaded using the proto-loader module. This file includes the ChatApi service definition with two methods: Connect and Broadcast. These definitions are then loaded into a chat object.
• Creates a gRPC client that connects to the specified server URL. It uses the ChatApi service defined in the chat.proto file for this connection.
• Establishes a server-side streaming gRPC connection to the server by using the Connect method passing in the client ID. The server will maintain this open stream and send messages to the client whenever a new chat message is available.
• Listeners are configured for the data, error, and end events of the stream. The data event handler logs the user’s ID and message content. If an error occurs or the server closes the stream, the error and end event handlers log a message and terminate the process.
• A listener is established on the line event of the readline interface, which is triggered when the user enters text in the console. The event handler sends a chat message to the server using the Broadcast method, including the client ID and message content.

That’s it, we have put together a chat service powered by gRPC, which effectively allows different programming languages to interact via the ChatApi service.

To extend our chat service app and provide a REST endpoint to retrieve all the connected clients from the server, we can make use of the grpc-gateway package from the grpc-ecosystem.

It enables us to create a reverse-proxy server that translates RESTful JSON API calls into gRPC calls. This allows us to have both a REST endpoint and a gRPC service.

To get the protoc-gen-grpc-gateway plugin, we can use the following command:

go get -u github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway

The first change we need to make is in our chat.proto file by introducing a new RPC method in the ChatApi service. We will name it GetClients, and it would return a array of client ids. The updated file would look like this:

syntax = "proto3";

package main;

option go_package = "./chat";

import "google/api/annotations.proto";

service ChatApi {
    rpc Connect (User) returns (stream Message);
    rpc Broadcast (Message) returns (Message);
    rpc GetClients (Empty) returns (ClientList) {
        option (google.api.http) = {
            get: "/clients"
        };
    };
}

message User {
    string id = 1;
}

message Message {
    User user = 1;
    string content = 2; 
}

message ClientList {
    repeated string client_ids = 1; // field can contain zero or more elements (like an array)
}

message Empty {}

The GetClients method doesn’t accept any parameters (Empty message) and returns a ClientList. The option (google.api.http) is a HTTP-to-gRPC mapping annotation, which specifies that this method can be invoked with a simple HTTP GET request to the /clients endpoint.

We are also importing annotations.proto which includes service configuration definitions used in HTTP mappings. The rules for these mappings are defined in a http.proto file.

To download these protobuf files, we can get them either directly from the official Google APIs GitHub repository or use the following curl commands:

curl https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/annotations.proto > proto/google/api/annotations.proto
curl https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/http.proto > proto/google/api/http.proto

Note: The files need to be in the following path: proto/google/api inside our project directory.

Next, we need to recompile our updated chat.proto file using protoc, same as before, but this time with some added options.

protoc --proto_path=proto proto/*.proto --go_out=gen/ --go-grpc_out=gen/ -I . \
  --grpc-gateway_out ./gen \
  --grpc-gateway_opt logtostderr=true \
  --grpc-gateway_opt generate_unbound_methods=true

The added options serves as follows:

1. -I: This includes the current directory in the search path, allowing protoc to locate any imports specified relative to the current directory.
2. --grpc-gateway_out ./gen: This tells the compiler to generate reverse-proxy gateway code using the grpc-gateway plugin and saves it to the ./gen directory. This code enables the translation of HTTP/JSON requests into gRPC calls and vice versa, allowing seamless communication between the two protocols.
3. --grpc-gateway_opt logtostderr=true: This option tells the grpc-gateway to log errors to the standard error output (stderr).
4. --grpc-gateway_opt generate_unbound_methods=true: This option tells the grpc-gateway plugin to generate gateway code for all methods in the service, even if they don’t have custom HTTP annotations. These unbound methods will be accessible via HTTP POST requests to a path that matches the RPC’s full method name, e.g. /package.Service/Method

Once the command is executed, a new file named chat.pb.gw.go will be generated in the gen/chat/ path. This file enables the exposure of our gRPC services over HTTP, making it possible to provide APIs that can be accessed by a broader range of clients, even those that do not have native support for gRPC.

The updated project structure would look like this:

.
├── proto
│   ├── chat.proto
│   └── google
│       └── api
│           ├── annotations.proto
│           └── http.proto
├── gen
│   └── chat // files generated by protoc will be stored here
│       ├── chat.pb.go
│       ├── chat_grpc.pb.go
│       └── chat.pb.gw.go
├── server
│   └── server.go
├── client
│   └── client.go
├── main.go
├── js-client
│   └── client.js

The next step would be update the server.go file to implement the GetClients method for our ChatApi service and to set up an HTTP server to host the REST API, which will act as a gRPC-HTTP gateway.

package server

import (
 "context"
 "log"
 "net"
 "net/http"
 "sync"
 "github.com/anik-ghosh-au7/grpc-messenger/gen/chat"
 "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
 "google.golang.org/grpc"
)

type server struct {
 chat.UnimplementedChatApiServer
 clients                         map[string]chat.ChatApi_ConnectServer
 mu                              sync.Mutex
}

func (s *server) Connect(user *chat.User, stream chat.ChatApi_ConnectServer) error {
 s.mu.Lock()
 s.clients[user.Id] = stream
 s.mu.Unlock()
 log.Println("Client Connected: ", user.Id)
 <-stream.Context().Done()
 s.mu.Lock()
 delete(s.clients, user.Id)
 s.mu.Unlock()
 log.Println("Client Disconnected: ", user.Id)
 return nil
}

func (s *server) Broadcast(ctx context.Context, message *chat.Message) (*chat.Message, error) {
 s.mu.Lock()
 defer s.mu.Unlock()
 for id, clientStream := range s.clients {
  if id != message.User.Id {
   if err := clientStream.Send(message); err != nil {
    log.Println("Error broadcasting message to", id, ":", err)
   }
  }
 }
 return message, nil
}

func (s *server) GetClients(ctx context.Context, empty *chat.Empty) (*chat.ClientList, error) {
 s.mu.Lock()
 defer s.mu.Unlock()
 clientIDs := make([]string, 0, len(s.clients))
 for id := range s.clients {
  clientIDs = append(clientIDs, id)
 }
 return &chat.ClientList{
  ClientIds: clientIDs,
 }, nil
}

func StartGrpcServer() error {
 s := &server{
  clients: make(map[string]chat.ChatApi_ConnectServer),
 }
 go func() {
  mux := runtime.NewServeMux()
  chat.RegisterChatApiHandlerServer(context.Background(), mux, s)
  log.Fatalln(http.ListenAndServe(":8000", mux))
 }()
 listener, err := net.Listen("tcp", ":8080")
 if err != nil {
  return err
 }
 srv := grpc.NewServer()
 chat.RegisterChatApiServer(srv, s)
 log.Println("Server started. Listening on port 8080.")
 return srv.Serve(listener)
}

The GetClients method will perform the following operations:

• Handles a *chat.Empty argument which serves as a placeholder since it does not require any real input parameters.
• Ensures safe concurrent access to the s.clients map.
• Creates a slice called clientIDs to hold the IDs of the connected clients.
• Loops over the keys in the s.clients map and appends each ID to the clientIDs slice.
• Returns a pointer to a chat.ClientList object that contains the clientIDs slice, and nil to indicate no error.

The updated StartGrpcServer function now starts a go-routine that creates a new ServeMux from the runtime package of grpc-gateway. This ServeMux acts as a multiplexer for routing HTTP requests to their respective handlers. The function then registers the gRPC gateway with this mux, starts an HTTP server, and begins listening on port 8000.

If the gRPC server is up and running with the HTTP gateway properly configured, and the GetClients method is implemented and mapped to the /clients path, executing the following command will give us back a valid JSON response.

curl http://localhost:8000/clients

The last step is to update our client.js to add an includeDirs option to packageDefinition. This instructs the protoLoader.loadSync method where to find imported .proto files.

const packageDefinition = protoLoader.loadSync('../proto/chat.proto', {
      ... all other fields
      // this is to load proto files from google, otherwise we would see error:
      // unresolvable extensions: 'extend google.protobuf.MethodOptions' in .google.api
      includeDirs: ['../proto/google/api', '../proto'],
    });

That’s it, we are finally done !!!

The gRPC framework is an excellent example of how technology can bridge language barriers, enhancing global communication and promoting a more interconnected world.

Note: The full source code can be found in this GitHub repository: https://github.com/anik-ghosh-au7/grpc-messenger.git