Interfacing GRPC (HTTP/2) to the Web Tier

Update (Aug 2018)- There is a better option than this ; there is a GRPC-Web project from Google; Please check this first https://medium.com/@alexcpn/interface-grpc-with-web-using-grpc-web-and-envoy-possibly-the-best-way-forward-3ae9671af67

There are two options in general for making your GRPC service accessible to services like browser side JavaScript that consume REST, or even API gateways.

Option 1

For GUI consumption,a better approach would be to have something like a Facade service written in the like of Backend for Front end pattern that mediates between grpc and web browser using Node.js based technology. We will see this later.

The advantage of this approach is that, with micro services there would be too much logic to orchestrate between services and an orchestration layer or service is going to come up somehow. Better to have this at the beginning and using TypeScript and Node.js technology this will be finally in the realm of front end developers.

They could write Typescript GRPC client and expose their own custom REST API for the GUI part, dealing with the changes and new versions of the other services in their own timelines. In Node.JS a web server serving this REST could be run for the front end.

Option 2

There is another way; a more automatic way of exposing a HTTP server with the GRPC interface directly via the grpc -web project, which is more elegant than the grpc gateway way; thanks to the team at Improbable OS for open sourcing this.

Let us start from the beginning;

Define a proto file for your service.

Let us take a very simple one,an Echo service, your_service.proto

syntax = "proto3";
package example;
message StringMessage {
string value = 1;
}
service YourService {
rpc Echo(StringMessage) returns (StringMessage) {
}
}

Now there are two options you can take;aren’t we spoiled for choices …

There are two choices with grpc-web as well

Option 1 : Implement the service in Go programming language.

If you do this, then you can use grpc-web Go module — to wrap the service and expose it as HTTP/1.1.

Let us do that;

Step 1: Generate the Grpc and protobuffer stubs from proto; For this (or any Grpc project) you need to install ProtoBuffer. Note that for grpc we need proto3 support; I used a Linux machine for development and had installed Protoc version 3 in it by compiling from sources as described here.

#protoc --version
libprotoc 3.2.0

Now create a simple folder structure for your project; I named the root folder test_proto and created a folder proto to keep the proto file and also the generated stubs. Now to generate the stubs

protoc -I proto/ proto/your_service.proto --go_out=plugins=grpc:proto

This will generate a your_service.pb.go

Now let us implement a very simple Go Server. I used VSCode and added Go language support. Since the source is simple I will list it and explain later. If you are starting with Go, then it is a language similar to C, with a strict import semantics (for faster builds than C/C++), simple and effective GC, pointers and interfaces/class as type; Some cons could be lack of a good dependency management, that is versioning concept of external libs are tied to latest of Git HEAD currently; and of course , new languages may lack the highly tuned libraries that are built over years examples the ones for C/C++ which are easily usable in Python, or the ones for Java that are easily usable for Scala etc. The main problems that Go was designed to solve are listed here.

Here is a simple Go GRPC service for the proto that exposes HTTP/2. This link here could be helpful http://www.grpc.io/docs/tutorials/basic/go.html

go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
go get -u github.com/golang/protobuf/protoc-gen-go
package main
import (
 "log"
 "net"
 pb "test_proto/proto"
"golang.org/x/net/context"
 "google.golang.org/grpc"
 "google.golang.org/grpc/reflection"
)
const port = ":7001"
type server struct{}
func (s *server) Echo(ctx context.Context, in *pb.StringMessage) (*pb.StringMessage, error) {
 log.Println("Got a request")
 return &pb.StringMessage{Value: "Hello From a GRPC Method (GRPC WEB Go Server Served) " + in.Value}, nil
}
func main() {
log.Println("Hello Go")
 lis, err := net.Listen("tcp", port)
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 s := grpc.NewServer()
reflection.Register(s)
 if err := s.Serve(lis); err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

Now let us transform the main part, wrap it in grpc-web so that it server HTTP/2

go get -u github.com/improbable-eng/grpc-web/go/grpcweb
import (
 "fmt"
 "log"
 "net/http"
 pb "test_proto/proto"
"github.com/improbable-eng/grpc-web/go/grpcweb"
 "golang.org/x/net/context"
 "google.golang.org/grpc"
 "google.golang.org/grpc/grpclog"
)
const port = ":7001"
type server struct{}
func (s *server) Echo(ctx context.Context, in *pb.StringMessage) (*pb.StringMessage, error) {
 log.Println("Got a request")
 return &pb.StringMessage{Value: "Hello From a GRPC Method (GRPC WEB Go Server Served) " + in.Value}, nil
}
func main() {
log.Println("Hello Go")
s := grpc.NewServer()
 pb.RegisterYourServiceServer(s, &server{})
 // wrapping in GRPC WEB
 wrappedServer := grpcweb.WrapServer(s)
 handler := func(resp http.ResponseWriter, req *http.Request) {
  wrappedServer.ServeHTTP(resp, req)
}
 httpServer := http.Server{
  Addr:    fmt.Sprintf(":%d", 8001),
  Handler: http.HandlerFunc(handler),
 }
if err := httpServer.ListenAndServe(); err != nil {
  grpclog.Fatalf("failed starting http server: %v", err)
 }
}

Run your Server (for HTTPS please see https://github.com/improbable-eng/grpc-web/tree/master/go/grpcweb)

go run test_server.go
2017/06/08 15:52:34 Hello Go

Now Let us Have a Client for this Service; And for that we use TypeScript with Node.JS and grpc_web Typesript Node.JS modules . TypeScript is a super-set of JavaScript offering static typing and well adopted; and TypeScript with Node.JS is a good combination for the Back End For Front End Pattern (we will see this option later). From the your_service.proto file, we need to create the TypeScript GRPC Stubs. You can follow the instructions here https://github.com/improbable-eng/ts-protoc-gen

npm install ts-protoc-gen

and generate the stubs

protoc --plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts --js_out=import_style=commonjs,binary:gene
rated --ts_out=service=true:generated -I /home/alex/go-path/src/test_proto/proto/ /home/alex/go-path/src/test_pro
to/proto/your_service.proto

Note that node.js packages most dependencies to ./node_moudles folder; For that you need to either install it yourself like

npm install -g typescript
 npm install @types/node --save-dev
 npm install ts-protoc-gen --> from grpc-web
 npm install grpc-web-client --> from grpc-web
 npm install google-protobuf --> had some problems with NPM
 npm install --save-dev typescript gulp gulp-typescript --> for gulp
 npm install --save-dev babelify

or better add the dependencies in Project package.json and use npm install

"devDependencies": {
    "@types/node": "^7.0.27",
    "babelify": "^7.3.0",
    "browserify": "^14.4.0",
    "gulp": "^3.9.1",
    "gulp-typescript": "^3.1.7",
    "tsify": "^3.0.1",
    "typescript": "^2.3.4",
    "vinyl-source-stream": "^1.1.0"
  },
  "dependencies": {
    "@types/google-protobuf": "^3.2.6",
    "google-protobuf": "^3.2.5",
    "typescript": "^2.3.4"
  }

If you are really wondering what Gulp is or why I added it, better to have a short read of Gulp; basically it is something like make.

The TypeScript has to be compiled to JavaScript so that it can be included in HTML and called from HTML; Instead of using raw tsc (TypeScript Compiler) to compile to JavaScript, if you have multiple JS modules, using browserify you can create a single bundle.js that you can use in HTML code. Gulp with the associated gulpfile.js takes care of automating the build flow.

Now let us create a simple node.js project and add the main Typescript file grpc_client.ts which calls the grpc-web wrapped service. I will put the code in git; But here is the gist

// A TypeScipt. WebClient for GRPC
// Based on https://github.com/improbable-eng/grpc-web/blob/master/example/ts/src/index.ts
import { grpc, BrowserHeaders } from "grpc-web-client";
import { YourService } from "../generated/your_service_pb_service"
import {StringMessage} from "../generated/your_service_pb"
function getResponse(){
        
    const stringMessage = new StringMessage();
    stringMessage.setValue("hello");
    
       grpc.invoke(YourService.Echo ,{
        request :stringMessage,
        host : "http://localhost:8001",
        onHeaders: (headers: BrowserHeaders) => {
            console.log("Echo.onHeaders", headers);
        },
        onMessage: (message: StringMessage) => {
            console.log("Echo.onMessage", message.toObject());
            const elt = document.getElementById("greeting");
            elt.innerText = message.toString();
        },
        onEnd: (code: grpc.Code, msg: string, trailers: BrowserHeaders) => {
            console.log("Echo.onEnd", code, msg, trailers);
        }
    })
    
}
getResponse();

now , write up a small html page that includes and invokes the JS

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8" />
    <title>Hello GRPC Web!</title>
</head>
<body>
    <p id="greeting">Loading ...</p>
    <script src="bundle.js"></script>
</body>
</html>

run gulp to create the bundle.js

gulp

Open the index.html in the dist folder that gulp has put

You should see

Here is the Code for Server and Client https://github.com/alexcpn/grpc-expose-web/

Now let us explore Option 2 with grpc-web. For this we will re use the same proto; we will change the server so that it exposes normal HTTP/2 like any other GRPC server in C++, Java etc

package main
import (
 "log"
 "net"
 pb "test_proto/proto"
"google.golang.org/grpc"
 "google.golang.org/grpc/reflection"
"golang.org/x/net/context"
)
const port = ":7001"
type server struct{}
func (s *server) Echo(ctx context.Context, in *pb.StringMessage) (*pb.StringMessage, error) {
 log.Println("Got a request")
 return &pb.StringMessage{Value: "Hello From a GRPC Method (GRPC WEB Go Server Served) " + in.Value}, nil
}
func main() {
log.Println("Hello Go GRPC")
 lis, err := net.Listen("tcp", port)
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 s := grpc.NewServer()
 pb.RegisterYourServiceServer(s, &server{})
 reflection.Register(s)
 if err := s.Serve(lis); err != nil {
  log.Fatalf("failed to serve: %v", err)
 }

Now if we invoke the client, we will get an error as it cannot directly query the HTTP/2 server

fetch.js:18 OPTIONS http://localhost:8001/example.YourService/Echo net::ERR_CONNECTION_REFUSED

We can use the grpc-web-proxy for this. You need Go version > = 1.7 for the above

:~/go-path/src/test_proto$ go version
go version go1.8.3 linux/amd64
 go get -u github.com/improbable-eng/grpc-web/go/grpcwebproxy

Start the GRPC Go server as before

go run test_server.go
2017/06/08 19:54:15 Hello Go GRPC
2017/06/08 19:59:58 Got a request

Run the grpc-proxy in another terminal;

Note — this grpcwebproxy is a simple project and has a bug that it is not working without TLS; I downloaded the source and commented out the check and build it a bit to make it work.

:~/go-path/src$ ./grpcwebproxy  --backend_addr=localhost:7001 --backend_tls=true
INFO[0000] listening for http_tls on: [::]:8443
INFO[0000] listening for http on: [::]:8080
INFO[0264] finished streaming call                       grpc.code=OK grpc.method=Echo grpc.service=example.YourService grpc.time_ms=1 span.kind=server system=grpc

And in the client TS changed to port to 8080 which the proxy is listening on and regenerated the bundle.js using gulp

grpc.invoke(YourService.Echo ,{
        request :stringMessage,
        //host : "http://localhost:8001",
        host: "http://localhost:8080",
Echo.onHeaders BrowserHeaders {keyValueMap: Object}
grpc_client.ts:20 Echo.onMessage Object {value: "Hello From a GRPC Method (GRPC WEB Go Server Served) hello"}
grpc_client.ts:25 Echo.onEnd 0 undefined