Get going with GRPC and Node.js | Part-2

In the last part we discussed at length about the two ways of employing GRPC with Node Js.

In this iteration, we will be implementing a sample project and working with the aforementioned methods. There are certain key points that need attention while approaching either of the methods discussed, which will be pointed out as we move along.

Defining our protobuf file

Assuming you have Node.js and NPM already present on your system, we can start by defining our protobuf file. We are going to go with a simple calculator project, which should result in a .proto file like the following:

// Specifies the syntax version of the protocol buffers used in this .proto file i.e. version 3
syntax="proto3";

package calculator;

// Service definition for the Calculator service
service Calculator {
  // RPC for adding two numbers
  rpc Add(AddRequest) returns (CalculatedResponse){}
  
  // RPC for multiplying two numbers
  rpc Multiply(MultiplyRequest) returns (CalculatedResponse){}
  
  // RPC for calculating factorial with streaming response
  rpc Factorial(FactorialRequest) returns (stream FactorialResponse){}
  
  // RPC for adding multiple numbers with streaming request and response
  rpc AddMultiple(stream AddRequest) returns (stream CalculatedResponse){}
}

// Message for the request of the Add RPC
message AddRequest {
    int32 num1 = 1;
    int32 num2 = 2;
}

// Message for the request of the Multiply RPC
message MultiplyRequest {
    int32 num1 = 1;
    int32 num2 = 2;
}

// Message for the request of the Factorial RPC
message FactorialRequest {
    int32 number = 1;
}

// Message for the response of the Factorial RPC
message FactorialResponse {
    int64 result = 1;
    int32 stage = 2;
}

// Message for the response of the Add and Multiply RPCs
message CalculatedResponse {
    int32 result = 1;
}

The protobuf defines 4 rpc methods under calculator service. i.e. the 4 rpcs define the operation calls between client and server group under the service called ‘Calculator’.

Each call defines the name, the input message type, an output message type along with the option to define them as streaming, and then we have our message definitions

Now that we have defined our proto files lets setup our server and client.

Static Generation

Let’s first install the package required for static generation:

`npm install -g grpc-tools`

`npm install google-protobuf`

Now you can generate your static code files with the following command

grpc_tools_node_protoc —protopath ./protos --js-out=import_style=commonjs,binary:./protosgen --grpc-out=./protosgen calculator.proto

Generating calculator_pb.js and calculator_grpc_pb.js in protosgen directory

Folder Structure For Static Generation:

static
├── protos
│   └── calculator.proto
├── protosgen
│   ├── calculator_pb.js
│   └── calculator_grpc_pb.js
├── client.js
├── server.js
├── package.json
├── package-lock.json
└── .gitignore

Now we will implement server and client stubs using these generated files

server.js

// Import the necessary dependencies
const grpc = require("@grpc/grpc-js");
const calculator = require("./protosgen/calculator_pb");
const services = require("./protosgen/calculator_grpc_pb");

// Create a new gRPC server
const server = new grpc.Server();

// Implementation of the 'add' gRPC method
const addImplementation = (call, callback) => {
  const num1 = call.request.getNum1();
  const num2 = call.request.getNum2();

  // Calculate the sum
  const result = num1 + num2;

  // Create a CalculatedResponse message for the sum
  const response = new calculator.CalculatedResponse();
  response.setResult(result);

  //Send calculated result back to client with no error
  callback(null, response);
};

// Add the implemented methods to the gRPC server
server.addService(services.CalculatorService, {
  add: addImplementation,
});

// Bind the gRPC server to a specific address and port, and start the server
server.bindAsync(
  "127.0.0.1:8080",
  grpc.ServerCredentials.createInsecure(),
  (err, port) => {
    if (!err) {
      console.log("Server started on port", port);
      server.start();
    } else {
      console.log("Error:", err);
    }
  }
);

Above we have first importing the generated files, making a server using the Server method provided by the ‘@grpc/grpc-js’ package. Then implementing the add method and further adding it as a service, then binding our grpc server to a specific address and server and starting it

Here I have only implemented a single rpc from our proto file. For complete implementation you can go to the github repository here

Now implementing the client stub for the same

client.js

const grpc = require("@grpc/grpc-js");
const calculator = require("./protosgen/calculator_pb");
const service = require("./protosgen/calculator_grpc_pb");

//Setting up the client stub
const client = new service.CalculatorClient(
  "127.0.0.1:8080",
  grpc.credentials.createInsecure()
);

// Function to request addition
const requestAddition = () => {
  const request = new calculator.AddRequest();
  request.setNum1(6);
  request.setNum2(6);

  client.add(request, (err, response) => {
    if (err) {
      console.log("error", err);
    } else {
      console.log("res", response.getResult());
    }
  });
};


// Call the functions to make requests
requestAddition();

When writing your client stubs you first setup the stub providing in the address and port where your grpc server is running and gRPC credentials. After that the each of the functions contain our request and then a call to the server stub where our request is passed. The response is also handled inside these functions through means of callbacks and listeners.

Running your server and client files you can exchange data between your client and server using gRPC.

Note: In this code sample, we are utilizing the @grpc/grpc-js library instead of the grpc library for our Node.js project. After generating your static files, it is necessary to update the imported library in your generated grpc file (e.g., calculator_grpc_pb.js) from grpc to @grpc/grpc-js, as shown below:

Dynamic Generation

Now lets go over the method where we load our proto file during runtime and implement data exchange using gRPC and the same .proto file

First lets install the packages required for dynamic loading of our proto files

npm install @grpc/proto-loader
npm install @grpc/groc-js

Folder Structure for dynamic generation:

dynamic
├── protos
│   └── calculator.proto
├── client.js
├── server.js
├── package.json
└── package-lock.json

We are going to first implement our server and then our client stub.

server.js

const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");

// Define the path to your proto file
const PROTO_PATH = "./protos/calculator.proto";

// Load the proto file using proto-loader
const packageDefinition = protoLoader.loadSync(PROTO_PATH, {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
});

// Get the package definition from the loaded proto file
const { calculator } = grpc.loadPackageDefinition(packageDefinition);

// Create your server using the dynamically generated package definition
const server = new grpc.Server();

// Implement your server logic using the dynamically generated package definition
server.addService(calculator.Calculator.service, {
  Add: (call, callback) => {
    const { num1, num2 } = call.request;
    const result = num1 + num2;
    const response = { result };
    callback(null, response);
  },
  Factorial: (call) => {
    // Implement your Factorial RPC logic here with streaming response
    const { number } = call.request;

    let result = 1;
    for (let i = 1; i <= number; i++) {
      // Calculate the factorial iteratively
      result *= i;

      // Create a response message for the intermediate result
      const response = {
        stage: i,
        result: result,
      };

      // Send the response to the client
      call.write(response);
    }

    // Signal the end of the stream
    call.end();
  },
});

// Start your server
server.bindAsync(
  "localhost:50051",
  grpc.ServerCredentials.createInsecure(),
  () => {
    server.start();
  }
);

Here we are dynamically loading our protofile using the proto-loader package and then storing the package definition inside the calculator variable. After that we create our gRPC server and add our service definitions. Alas, we bind our server to an address and start it so that we can start sending our requests.

client.js

const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");

// Define the path to your proto file
const PROTO_PATH = "./protos/calculator.proto";

// Load the proto file using proto-loader
const packageDefinition = protoLoader.loadSync(PROTO_PATH, {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
});

// Get the package definition from the loaded proto file
const { calculator } = grpc.loadPackageDefinition(packageDefinition);

// Create your client using the dynamically generated package definition
const client = new calculator.Calculator(
  "localhost:50051",
  grpc.credentials.createInsecure()
);

// Call your RPCs using the dynamically generated package definition
client.Add({ num1: 1, num2: 2 }, (err, response) => {
  if (err) {
    console.error(err);
  } else {
    console.log("Add Response:", response);
  }
});



const factorialCall = client.Factorial({ number: 5 });

factorialCall.on("data", (response) => {
  console.log("Factorial Response:", response);
});

factorialCall.on("error", (err) => {
  console.error(err);
});

factorialCall.on("end", () => {
  console.log("Factorial Streaming Complete");
});

Here in our client.js, similar to server.js we dynamically load the proto file and then extract the package definition by the name of the package defined inside the .proto file. Then, we create our client to make calls to our server where we pass the address our server is running on.

After this, we can make calls and handle incoming data from those calls using our client.

I have only shown two rpc methods here, for complete implementation you can visit the github repository here

Note: When working with gRPC in JavaScript, it’s important to note that the usage of getters and setters may differ between static code generation and dynamic code generation methods. In static code generation, where the protocol buffers are pre-compiled into static JavaScript files, you can use explicit getter and setter methods to access and modify the message fields. However, in dynamic code generation, where the protocol buffers are dynamically loaded at runtime, the gRPC call object’s methods are typically used to send and receive messages, and explicit getter and setter methods may not be available. In the example provided, we showcase the usage of getters and setter methods in the static generation method (using pre-compiled JavaScript files) but not in the dynamic generation method (using grpc.loadPackageDefinition() ).

This is because the call object provided by the gRPC library acts as the context for the ongoing call and provides methods to interact with the client, without implementing actual JavaScript getter and setter syntax.

Conclusion

In conclusion, gRPC is a powerful and efficient remote procedure call (RPC) framework that allows seamless communication between microservices in distributed systems. In this article, we explored how to work with gRPC in Node.js, covering two different approaches — static generation and dynamic generation. By following the examples and guidelines provided in this article, you can now get started with gRPC in Node.js and leverage its benefits. Whether you choose static generation or dynamic generation, it’s important to understand the trade-offs and choose the approach that best fits your project requirements.

I have only shown two rpc methods here, for complete implementation you can visit the github repository here