Unleashing gRPC Gateway in Golang: Crafting a Reverse Proxy for Effortless RESTful Integration

9 min readNov 18, 2023

Introduction

Greetings, fellow enthusiasts of Golang, gRPC, and the powerful ScyllaDB! Our recent exploration marked not the end but a new beginning in the vast landscapes of these technologies. The possibilities are limitless, and as we stride forward, the next chapter beckons with exciting prospects.

Continuing the Journey

Continuing our journey from the intersection of Golang, gRPC, and ScyllaDB, we venture deeper into uncharted territories. This time, we unravel the art of seamlessly integrating a RESTful JSON API with the stalwart gRPC architecture. Welcome to the second chapter of our saga, where innovation knows no bounds.

A Sneak Peek

Today, we embark on a mission to fuse an HTTP+JSON interface with our gRPC service, unveiling a world where minimal configuration adjustments lead to the effortless generation of a reverse proxy. This endeavor not only enhances the adaptability of your service but sets the stage for catering to an even broader spectrum of application scenarios.

Today, we unravel the art of modifying our beloved protobufs to seamlessly integrate a RESTful JSON API with the robust gRPC architecture. Buckle up as we explore the transition from the familiar to the extraordinary, using the same old proto but with a touch of innovation.

Set Up for Grpc Gateway

Install the grpc gateway packages by running the below command:

go install \
    github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway \
    github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2 \
    google.golang.org/protobuf/cmd/protoc-gen-go \
    google.golang.org/grpc/cmd/protoc-gen-go-grpc

This command ensures the availability of key tools:

protoc-gen-grpc-gateway: Generates Go code for gRPC Gateway, facilitating communication between gRPC and RESTful APIs.
protoc-gen-openapiv2: Enables automatic creation of OpenAPIv2 specifications for gRPC services, enhancing API documentation.
protoc-gen-go: Generates Go code from Protobuf definitions, allowing seamless integration of Protobuf structures in Go programs.
protoc-gen-go-grpc: Creates Go code to support gRPC communication for a given Protobuf file, enabling the development of gRPC servers and clients in Go.

Exploring the Proto Transformations: A Deep Dive

In the dynamic landscape of gRPC, understanding the role of reverse proxies is pivotal. These gatekeepers hold the key to seamless communication between gRPC services and the diverse world of HTTP clients. In this exploration, we unravel the purpose of a reverse proxy in the gRPC context, shedding light on its profound benefits. Additionally, we’ll introduce the game-changing google.api.http option and uncover how it orchestrates HTTP-specific configurations for each gRPC service method.

Section 1:

Unveiling the google.api.http Option: Enter the game-changer — the google.api.http option. This option, when integrated into your protobuf definitions, empowers gRPC services to expose HTTP-specific configurations. Let's delve into its intricacies:

Purpose and Usage: The google.api.http option serves as a declarative way to define how each gRPC service method should be exposed over HTTP. By embedding this option in your proto file, you dictate how clients can interact with your gRPC service through HTTP.
Syntax: The syntax involves specifying HTTP methods, paths, request and response body configurations, and more, providing a comprehensive blueprint for translating gRPC operations into HTTP endpoints.

Section 2:

Transforming gRPC Methods into HTTP Endpoints: Concrete examples bring concepts to life. Let’s explore how specific gRPC methods undergo a metamorphosis, emerging as HTTP endpoints:

Example 1: Creating a Movie — From gRPC to HTTP:

rpc CreateMovie(MovieRequest) returns (MovieResponse) {
    option (google.api.http) = {
        post: "/api/V1/create/movie"
        body:  "*"
    };
}

In this snippet, the CreateMovie gRPC method is transformed into an HTTP POST endpoint at the path /api/V1/create/movie.

Example 2: Getting All Movies — A GET Endpoint Emerges:

rpc GetAllMovies(GetAllMoviesRequest) returns (GetAllMoviesResponse) {
    option (google.api.http) = {
        get: "/api/V1/get/all/movies"
    };
}

The GetAllMovies method effortlessly transitions into an HTTP GET endpoint at /api/V1/get/all/movies.

Once the changes are done in proto files, compile the proto files into Go files with gRPC support, use below command

protoc --proto_path= proto/*.proto --go_out=resources/ --go-grpc_out=resources/ --grpc-gateway_out=resources/

--proto_path=proto/: Specifies the directory where your Proto files are located. Update this path based on your project's structure.
--go_out=resources/: Indicates the output directory for the generated Go files. The resources/ directory will be created in the same location as your proto/ directory.
--go-grpc_out=resources/: Specifies the output directory for the generated gRPC Go files. The resources/ directory will be created in the same location as your proto/ directory.
proto/*.proto: Specifies the Proto files to be compiled. Adjust this pattern based on the actual names and locations of your Proto files.
--grpc-gateway_out=resources/: Generates Go files for gRPC Gateway, which allows your gRPC service to be accessible via RESTful HTTP. The generated files are placed in the resources/ directory.

Creating a Foundation: The Router Structure

The role of an HTTP router is pivotal. In this section, we explore the creation of a versatile HTTP router, utilizing the Gorilla Mux package. From dynamic routing to health checks and profiling.

http/router.go

Initiating the Router

type RouteConfig struct {
    Path    string
    Handler http.Handler
    Methods []string
}

type Router interface {
    AddRoute(config RouteConfig)
    StaticRoute()
    Mux() http.Handler
}

type router struct {
 mux *mux.Router
 sync.Mutex
}

func NewRouter() Router {
    r := &router{
        mux: mux.NewRouter().StrictSlash(true),
    }
    r.debugRoutes()
    r.healthCheckRoute()
    return r
}

At the heart of our HTTP router is a well-defined structure. RouteConfig captures essential details for each route, while the Router interface outlines the core methods, including adding routes, defining static routes, and obtaining the underlying HTTP handler.

Creating an instance of our router involves initializing a mux.Router. This sets the stage for further customization and dynamic routing. Additionally, we establish default routes for debugging and health checks.

Adding Routes Dynamically

func (r *router) AddRoute(route RouteConfig) {
    r.Lock()
    defer r.Unlock()
    handler := route.Handler

    rt := r.mux.Handle(route.Path, handler)
    if route.Methods != nil && len(route.Methods) > 0 {
        rt.Methods(route.Methods...)
    }
}

The AddRoute method empowers dynamic route addition. It supports various HTTP methods, providing flexibility in configuring our router based on specific service requirements.

Enabling Static File Serving

func (r *router) StaticRoute() {
    mux := r.mux
    mux.PathPrefix("/").Handler(http.FileServer(http.Dir("/tmp/static")))
}

Our HTTP router goes beyond dynamic routes, offering static file serving capabilities. The StaticRoute method designates a path prefix for serving static files, enhancing the versatility of our router.

Accessing the Underlying Mux

func (r *router) Mux() http.Handler {
    return r.mux
}

The Mux method provides a seamless way to access the underlying mux.Router. This feature enables integration into larger applications and middleware, ensuring a smooth development experience.

Unlocking Debugging and Profiling

func (r *router) debugRoutes() {
    mux := r.mux
    mux.HandleFunc("/debug/pprof/", pprof.Index)
    mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
    mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
    mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
    mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
    mux.Handle("/debug/pprof/goroutine", pprof.Handler("goroutine"))
}

Our HTTP router is equipped with debugging and profiling capabilities. The debugRoutes method adds routes for Go's pprof package, allowing developers to gain insights into performance characteristics.

Health Checks Made Easy

func (r *router) healthCheckRoute() {
    r.mux.HandleFunc("/health_check", healthCheck)
}

func healthCheck(w http.ResponseWriter, req *http.Request) {
    w.Write([]byte("Health Check Passed"))
}

Ensuring the health of our services is paramount. The healthCheckRoute method adds a straightforward health check endpoint, responding with a reassuring "Health Check Passed" message.

Unveiling the Server Structure

http/server.go

Crafting a robust HTTP server is key to unlocking the full potential of your services. In this segment, we delve into the implementation of a feature-rich HTTP server, seamlessly integrating with the HTTP router. From CORS configuration to graceful shutdowns, join us in exploring the intricacies of our Go HTTP server

Initializing the Server

type Server struct {
    *zap.Logger
    config *config.AppConfig
    srv    *http.Server
    mux    http.Handler
}

func NewServer(config *config.AppConfig, router Router) *Server {
    return &Server{
        config: config,
        mux:    router.Mux(),
        Logger: log.Logger,
    }
}

At the core of our HTTP server lies a well-architected structure. The Server type integrates a Zap logger, application configurations, an HTTP server instance, and the router's multiplexer, setting the stage for a harmonious collaboration.
The NewServer function serves as the gateway to our HTTP server, bringing together application configurations and the router. This function creates and returns a pointer to a configured Server instance.

Commencing the Server

func (s *Server) Start() {
    srv := &http.Server{
        Handler:      handlers.CORS(handlers.AllowedHeaders([]string{"X-Requested-With", "Content-Type", "Authorization"}), handlers.AllowedMethods([]string{"GET", "POST", "PUT", "HEAD", "OPTIONS"}), handlers.AllowedOrigins([]string{"*"}))(s.mux),
        ReadTimeout:  s.config.HttpReadTimeout,
        WriteTimeout: s.config.HttpWriteTimeout,
        IdleTimeout:  s.config.HttpIdleTimeout,
    }
    s.srv = srv

    err := srv.ListenAndServe()
    if err != nil && !strings.Contains(err.Error(), "http: Server Closed") {
        s.Logger.Fatal("failed to start server %v", zap.Error(err))
    }
}

Embarking on the server journey is the Start method. It configures CORS support, sets up timeouts, and initiates the HTTP server. The graceful handling of errors ensures a resilient and reliable server start-up.

HTTP Initialization: A Symphony of Components

main.go

func initHttpModules(router http.Router, cfg *config.AppConfig) {
    log.Logger.Info("adding static asset route")
    router.StaticRoute()
    server := http.NewServer(cfg, router)
    log.Logger.Info("HTTP Server running")
    go server.Start()
}

server := http.NewServer(cfg, router): Creates a new HTTP server instance using the NewServer function from the http2 package. This function takes the application configuration (cfg) and the configured router.
go server.Start(): Launches the HTTP server concurrently using a goroutine, allowing it to handle incoming HTTP requests independently.

Gateway Integration

func runGatewayServer(configurations *config.AppConfig, movieServer *grpcserver.MovieGrpcServer) {
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()

    grpcMux := runtime.NewServeMux(
        runtime.WithMetadata(func(c context.Context, req *nethttp.Request) metadata.MD {
            return metadata.Pairs("x-forwarded-method", req.Method)
        }))
    opts := []grpc2.DialOption{grpc2.WithTransportCredentials(insecure.NewCredentials())}
    grpcServerEndpoint := flag.String(grpcServerEndpointName, constants.ColonSeparator+configurations.GRPCPort, grpcEndpointNameUsage)
    if err := moviepb.RegisterMoviePlatformHandlerFromEndpoint(ctx, grpcMux, *grpcServerEndpoint, opts); err != nil {
        log.Logger.Fatal("cannot register gateway handler server")
    }
    if err := nethttp.ListenAndServe(constants.ColonSeparator+configurations.ReverseProxyHttpPort, grpcMux); err != nil {
        log.Logger.Fatal("Failed to serve to", zap.String("port", configurations.ReverseProxyHttpPort), zap.Error(err))
    }
}

ctx, cancel := context.WithCancel(context.Background()): Creates a new context for the gateway server, allowing cancellation to be propagated through the cancel function.
grpcMux := runtime.NewServeMux(...): Initializes a new ServeMux for the gRPC gateway using the runtime package. This serves as the multiplexer for handling gRPC requests translated to HTTP.
opts := [...]: Defines gRPC dial options, including using insecure transport credentials.
grpcServerEndpoint := flag.String(...): Retrieves the gRPC server endpoint from command-line flags, specifying the address and port.
moviepb.RegisterMoviePlatformHandlerFromEndpoint(...): Registers the gRPC gateway handler, translating gRPC calls to HTTP requests.
http.ListenAndServe(...): Initiates the HTTP server for the gRPC gateway, serving on the specified port. Any errors during server startup are logged, and fatal errors lead to the termination of the application.

These methods collectively lay the foundation for HTTP integration and gRPC gateway functionality within your Go application. The orchestration of routers, servers, and gateway registration enables a cohesive and performant HTTP service.

Deploying and Testing the Http Endpoints

Build and Run the Application: Execute go build to compile the Golang code, preparing the binary for deployment. Once compiled, initiate the application using either of the following commands:

go build

go run main.go
    or
./go-grpc-service

Endpoint Verification

Let’s validate each accessible HTTP endpoint in our service. Begin by triggering the creation of a movie, storing its details in the database. Subsequently, retrieve the same information using a GET endpoint to ensure its accuracy. Finally, assess the update endpoint to confirm the successful modification of data in the datastore.

Conclusion: Unlocking the Potential of gRPC Gateway in Golang

As we bring this exploration of gRPC Gateway in Golang to a close, the horizon of possibilities expands. The journey doesn’t conclude here; it marks a new beginning in the realm of seamless integration. We’ve delved into the intricacies of building a reverse proxy, seamlessly blending RESTful HTTP APIs with the robust gRPC architecture.

In our quest for a versatile and adaptable solution, we embarked on a journey that unveiled the power of gRPC Gateway. By leveraging the Google protocol buffers compiler protoc, we transformed routine service definitions into endpoints capable of generating a reverse proxy server. This transformation facilitates the translation of RESTful HTTP APIs into the language of gRPC, offering a unified and efficient communication mechanism.

The journey included the modification of protobuf service definitions, introducing the magic of google.api.http options. These adjustments allowed us to effortlessly bridge the gap between the HTTP and gRPC worlds, enabling a smooth integration that caters to diverse application requirements.

As we conclude, this exploration invites continuous innovation. Embrace these technologies, contribute to the evolving landscape, and shape the future of microservices. The journey doesn’t end; it marks a new chapter in modern application development. Happy coding!

GitHub Link: https://github.com/Jitender271/go-grpc-service