How to use gRPC and protobuf in nodeJS microservice?

6 min readApr 28, 2019

In a long time, we are using REST service for sharing data between client and server. We are using the HTTP/1 protocol for transmitting data in textual format with encryption.

HTTP needs various metadata to specify the type of data, type of compression and many more headers. We need to sent additional data like header, status, type of compression and datatype. Due to HTTP/1.1 transmits textual data we are always using some serialization method (JSON/XML) to make the data parsable and human readable.

Why protobuf?

Protobuf is the new trending serialization method prepared by Google which acts as a serialization method in transmission medium between the HTTP transport layer. The most special thing in protobuf serialization is HTTP/2.

Google is developing technology to make the web faster. Protobuf is designed by Google which means it will work as a protocol buffer to transmit data only in buffers.

Once go through the official documentation of protobuf by google.

https://developers.google.com/protocol-buffers/docs/proto3

Google designed a language proto3 to make the serialization easier. I am not going to explain the proto3 language. If you are unaware of proto3 then once go through the language guide by Google here

Why HTTP/2?

It is a new protocol and faster than HTTP/1.1 protocol. It transmits data in binary, not in the textual format is the main reason behind the error-free transmission.

It is highly reliable to transfer data between the server and the client.
Single connection for multiple files: It can able to transfer multiple files in a single request.
Request and response streaming: Most powerful feature of HTTP/2 is request and response streaming which can able to send multiple requests and receive multiple responses in a single connection like a stream. (I will explain more about request and response streaming in this article with some interesting examples).
Go through the link https://imagekit.io/demo/http2-vs-http1 to see how faster the HTTP/2 protocol is.

gRPC

The real world is moving towards the Service Oriented Architecture (SOA) to making the infrastructure more scalable. We are going to use gRPC technology to making the remote connection because of it really faster than any other service like REST, SOAP and socket. If you are unaware of gRPC then once go through the official documentation of gRPC it supports many languages you can find the language guide here .

Here are some brief ideas about proto3 language.

Let’s make a simple request model that contains the user’s name in string type.

user.proto

Save this file as user.proto. Now the proto request model is ready to compile.

We are going to use some NPM packages to load the proto file internally. If You need to compile manually you can use protoc compile to compile the proto file.

Let’s create a simple welcome application which handles request and response.

In this application, I am going to send only the user’s name in the request and will receive the response as ‘Hello <User> Welcome to the world’.

Create a directory named ‘welcome-world’, and go to the directory in terminal.
Hit ‘npm init’ and follow the instructions to initialize the project.
inside that create a proto directory where the proto files will be stored.
Create a server directory where the server files will be stored and create a client directory where the client files will be stored.

Finally, your directory structure should look like this

Let’s create service inside the .proto file to handle requests and responses. Insert below code in the welcome-world.proto file

service WelcomeService {
 // this service method needs to be implemented.
 rpc greetUser(WelcomeRequest) returns (WelcomeResponse) {};
}

Your welcome-world.proto will be

welcome-world.proto

Here I have created a service named Welcome service which will contain our RPC methods.

I have created a simple RPC method named greetUser which accepts the request as message type “WelcomeRequest” and returns message type “welcome response”.This is the declaration of the RPC method we need to implement it inside our server. Let’s implement the service in our nodeJS application. Write the code below in your server.js file.

server.js

Here we have created a gRPC server in which we have implemented the greetUser RPC method which returns the response inside the callback.

The next step is to create a client to hit the service through the proto file.

Your client.js file will be

Run the server file $ node server.js

Run the client file $ node client.js

We have created a simple client-server application using gRPC and protobuf.

Next I am going to explain what is request-response streaming.

Request response streaming!

Request response streaming means you can able to send multiple requests and receive multiple responses in one TCP connection. It will improve your performance of the application while exchanging data through the web.

Here two services sharing their resources through gRPC in proto serialized contents, here gRPC opens a streaming connection between the services so that multiple request and responses can be shared within one connection.

gRPC opens a writable stream to write multiple responses.

We will make a deep dive into the requests and responses streaming in the next example.

Here I have created a chat service and four clients for the chat group.

The main idea of the application is that the service will create a chat group and the four clients can chat with each other in that chat group through gRPC stream.

Step-1

Create a directory named ‘chat-group’ which will contain our main chat server and inside that create a proto and server directory.