Building High-Performance Microservices with Node.js, gRPC, and TypeScript

A tutorial on using Node.js, gRPC, and TypeScript together to build high-performance microservices.

Node.js is a popular runtime environment for building scalable server-side applications. gRPC is an open-source high-performance Remote Procedure Call (RPC) framework that allows you to build efficient and scalable microservices. TypeScript is a typed superset of JavaScript that adds optional static typing to your code, making it more maintainable and easier to refactor. In this article, we will explore how to use Node.js, gRPC, and TypeScript together to build high-performance microservices.

Setting up a new project

To start, let’s create a new Node.js project and install the required dependencies. We will use the npm package manager to install the dependencies, so make sure you install it on your system. Run the following commands to create a new project and install the dependencies:

mkdir my-grpc-service
cd my-grpc-service
npm init -y
npm install grpc @grpc/proto-loader grpc-tools typescript ts-node-dev

This will create a new Node.js project with the required dependencies installed. We will use the grpc-tools package to generate TypeScript typings for our gRPC service.

Creating a gRPC service

Let’s create a simple gRPC user service

Create a user.proto file.

syntax = "proto3";

package user;

service UserService {
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
  rpc CreateUser(CreateUserRequest) returns (CreateUserResponse);
}

message GetUserRequest {
  string id = 1;
}

message GetUserResponse {
  string name = 1;
  string email = 2;
}

message CreateUserRequest {
  string name = 1;
  string email = 2;
}

message CreateUserResponse {
  string id = 1;
}

This user.proto the file defines a message called User with three string fields: id, name, and email. It also defines a service called UserService five RPC methods: GetUser, CreateUser. Each RPC method has a corresponding request message and response message, which are defined using the message keyword.

The syntax field at the top of the file specifies that this is a Protocol Buffers 3 file.

The next step is to generate the TypeScript

Wait! Why do we need to generate types?

Generating TypeScript files for gRPC is useful for a few reasons:

1. Strongly typed APIs: TypeScript provides static type checking which helps catch errors at compile-time rather than run-time. This helps ensure that the API client and server are communicating correctly.
2. Code completion and documentation: The TypeScript definitions for a gRPC API provide code completion and documentation in code editors, making it easier to work with the API.
3. Better maintainability: By generating TypeScript code from protobuf definitions, you can ensure that changes made to the API’s message types are reflected in both the client and server implementations, reducing the risk of errors and making maintenance easier.
4. Interoperability: By using the same definitions for both the client and server, you can ensure that they are using the same message formats, making it easier to integrate with other systems that use the same definitions.

But why exactly do we need protobuf?

protocol Buffers (protobuf) is a language-agnostic binary serialization format that is used to serialize structured data. When working with gRPC, the messages sent between the client and server need to be serialized and deserialized. This is where protobuf comes into play.

By defining the messages in a .proto file and using the Protocol Buffer compiler (protoc) to generate the corresponding code, we can easily generate classes and methods for working with these messages in multiple programming languages. This makes it easier to work with structured data across different platforms and languages.

So, we need to generate the protobuf files to define the messages that are exchanged between the client and server and also to generate the corresponding code to work with those messages in our chosen programming language.

A Common error

If you’re getting a “Missing input file” error when running the grpc_tools_node_protoc command, it's likely because the command can't find your .proto file. Here are a few things you can try:

1. Double-check the path to your .proto file: Make sure the path you're providing to the --proto_path the flag is correct, and the .proto file exists in that location.
2. Specify the .proto file directly: Instead of specifying a directory with the --proto_path flag, try specifying the .proto file directly using the --proto_files flag. For example:

grpc_tools_node_protoc --plugin=protoc-gen-ts=/usr/local/bin/protoc-gen-ts --ts_out=./ --proto_files=path/to/your/proto/file.proto

3. Install the grpc package: Make sure you have the grpc package installed in your project. You can install it using npm install grpc.

4. Use an absolute path: Instead of using a relative path to your .proto file, try using an absolute path.

grpc_tools_node_protoc --plugin=protoc-gen-ts=/usr/local/bin/protoc-gen-ts --ts_out=./ --proto_path=/absolute/path/to/your/proto/file.proto

My personal choice is a simple bash script.

#!/bin/bash

# Generate the JavaScript and TypeScript files from the proto file
./node_modules/.bin/grpc_tools_node_protoc \
  --js_out=import_style=commonjs,binary:./src/proto \
  --grpc_out=./src/proto \
  --plugin=protoc-gen-grpc=./node_modules/.bin/grpc_tools_node_protoc_plugin \
  --ts_out=./src/proto \
  ./src/proto/user.proto

# Rename the generated files to use camelCase instead of snake_case
for file in ./src/proto/*_pb.js; do mv "$file" "${file//_pb.js/_pb.js}"; done
for file in ./src/proto/*_grpc_pb.js; do mv "$file" "${file//_grpc_pb.js/_grpc_pb.js}"; done

Note: You can also use https://buf.build/docs/installation/ to generate types.

Define Server

import * as grpc from '@grpc/grpc-js';
import { GetUserRequest, GetUserResponse, CreateUserRequest, CreateUserResponse } from '../generated/user_pb';
import { UserService, IUserServer } from '../generated/user_grpc_pb';

function getUser(_call: grpc.ServerUnaryCall<GetUserRequest,GetUserResponse >, callback: grpc.sendUnaryData<GetUserResponse>): void {
  const userResponse = new GetUserResponse();
  userResponse.setName('John');
  userResponse.setEmail('john@example.com');

  callback(null, userResponse);
}

function createUser(_call: grpc.ServerUnaryCall<CreateUserRequest, CreateUserResponse>, callback: grpc.sendUnaryData<CreateUserResponse>): void {
  const userResponse = new CreateUserResponse();
  userResponse.setId('123');

  callback(null, userResponse);
}

const server = new grpc.Server();

server.addService(UserService, { getUser, createUser });

const port = process.env.PORT || '50051';

server.bindAsync(`0.0.0.0:${port}`, grpc.ServerCredentials.createInsecure(), () => {
  console.log(`Server running on port ${port}`);
  server.start();
});

This code defines two gRPC methods getUser and createUser for the UserService service. getUser takes a GetUserRequest object and returns a GetUserResponse object, while createUser takes a CreateUserRequest object and returns a CreateUserResponse object. The server is started on port 50051 with insecure credentials.

Create a simple client

import * as grpc from '@grpc/grpc-js';
import { UserServiceClient } from '../generated/user_grpc_pb';
import { CreateUserRequest, CreateUserResponse, GetUserRequest, GetUserResponse } from '../generated/user_pb';

const client = new UserServiceClient('localhost:50051', grpc.credentials.createInsecure());

const createUser = (name: string, email: string) => {
  const request = new CreateUserRequest();
  request.setName(name);
  request.setEmail(email);

  client.createUser(request, (err, response: CreateUserResponse) => {
    if (err) {
      console.error(err);
      return;
    }
    console.log(`User created with id: ${response.getId()}`);
  });
};

const getUser = (id: string) => {
  const request = new GetUserRequest();
  request.setId(id);

  client.getUser(request, (err, response: GetUserResponse) => {
    if (err) {
      console.error(err);
      return;
    }
    console.log(`User name: ${response.getName()}, email: ${response.getEmail()}`);
  });
};

createUser('Tuhin Banerjee', 'tuhinb@example.com');
getUser('123456');

Github repo for playing with the above code:

GitHub - geektuhin123/grpc-user-service: User Service to demonstrate the use of gRPC in Node.js

What’s next?

Are you considering building a high-performance distributed system? If so, you may be weighing the pros and cons of using a message queue versus a gRPC-based system. In my next article, we will dive into the details of these two approaches and compare their performance using BullJS, a popular Node.js library for handling background jobs. We’ll explore the strengths and weaknesses of both message queues and gRPC-based systems, and discuss the factors that you should consider when choosing between the two.