Writing simple unary RPC for computer vision applications with gRPC in Python
Writing a scalable application needs a better design principle. Let’s say we want to design a e-Commerce application, it may have a different set of clients (browser, mobile device, etc.), also in the middle of the development stage a deep-learning based recommendation system which works on images to extract product information needs to be written to improve the user experience, but the main application was written in PHP. To make the design as modular as possible, it’s better to choose a microservices based design strategy instead of writing the complete application as one cohesive unit, sharing the same memory space (monolith).
If we want to extract car models from image data, we can solve the task with two steps, first, we localize the cars with semantic segmentation, finally, for each segmented car, we apply a classification model to get the car model. A simple scenario with microservices architecture would be, team 1 is working on the segmentation model which they developed and deployed with microservice 1. Another team has developed the classification model and deployed their solution as 2nd microservice (overly simplified example).
The client first calls the microservice 1 to get the segmentation map, then uses the map to call microservice 2 and get the car model.
In an ideal scenario, we’ll have an API gateway, but here we’ll focus on very simple server-client API only.
In simple terms, the client will send a request and the server will return a response. This simple communication between services makes the whole thing work even though each block runs separately. Microservices can call other microservices internally to perform some tasks too.
REST + JSON
The most common tradition is to use REST for microservices. With REST, services communicate directly and synchronously with each other over HTTP. Usually, JSON is used as payload data type while accepting or responding.
A simple example
In computer vision, we’re mostly working with images. Let’s try to pass an image and some meta-data (width, height of the image) as request and get the average pixel value and number of channels in that image as our dummy task.
We can implement this with rest API with flask-restful in python.
The server accepts a post request with json data containing a base64 image, width (int), height (int); finds the mean; returns the mean, and the number of channels.
gRPC
The same result can be achieved with gRPC. gRPC uses a concept called remote procedure call (RPC). In simple words, it’s like executing a procedure in a remote address space as if it was a local procedure. In reality, everything happens on the network not locally, but with gRPC introducing protocol buffers (proto3) have become much more efficient. It uses binary data format which reduces network overhead and HTTP/2 makes bidirectional streaming possible.
To get started, first, we have to write the .proto file in protocol buffer language to structure our protocol buffer data.
We define two messages, one for the input request which will consist of a base64 image (string), the height (int), and the width (int). The response message will consist of one integer channel, and one float mean. We only need one service which will take a B64Image message, apply rpc ImageMeanWH, and return the Prediction message.
To use grpc with python (C-bindings):
pip install grpcio
pip install grpcio-toolsWe can compile the .proto file to generate the required python classes.
python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. image_procedure.protoThere will be two python files generated: image_procedure_pb2.py and image_procedure_pb2_grpc.py
In the client-side, we’ll create a stub, encode the image as base64 string and generate the request message with B64Image from the generated class. We will call the ImageMeanWH to get the response.
The server will internally call a predict function which will perform the required operation on the image. We can write predict as a function in a separate image_procedure.py file. Our predict function can be anything from calculating the mean of an image to running an object detection model on the image to get the bounding boxes.
Finally, this is how our predict function looks like for this dummy task.
A rough time comparison between flask-restful and grpc python (for 1000 requests) is shown below. (The numbers are a rough estimate; server, client are run in the same machine; for low network bandwidth zlib compression can improve the latency but for regular cases, it doesn’t reduce time)
The source-code is available here: https://github.com/zabir-nabil/simple-gRPC
