Reactive Microservices with Golang, Rabbitmq and Protobuf
We can build reactive, asynchronous and cross language microservices with
Protobuf stack. In this post I’m gonna discuss about building microservices based document management service with
Golang by using Rabbitmq and Protobuf. Protobuf use to serialized structured data(request response between microservices). Rabbitmq uses as message broker. The architecture of the system described in following figure.
There are two main services in the system,
Storage. Gateway service act as microservices api-gateway. Storage service do actual document management functions(ex: create documents in database). Client submit
JSON documents to gateway service via REST api. Gateway service create Protobuf message based on the JSON data and pass it to storage service via Rabbitmq. Then storage service do what ever the operation(ex: create documents on a database) and sends the status back to gateway service as Protobuf message. Finally gateway service convert Protobuf message which receives from storage service into JSON object and send it back to client. Gateway service listen to Rabbitmq
gateway queue and Storage service listen to
storage queue. All the source code which related to this post available on gitlab. Please clone the repo and continue the post.
document.proto defines the Protobuf structures of document object and request response objects. Main thing to notice here is
go_package="gitlab.com/rahasak-labs/rabbit/spec". When compiling the protobuf with
protoc it generate the go file in
Following is the way to compile the Protobuf spec. It will generate output go file inside
Gateway service start REST api on
/api/v1/documents:7654. When request comes to this api, it creates
CreateDocumentMessage and forward it to storage service.
Rabbitmq publisher implemented with
publisher.go. It gets the messages from go channel and publish to Rabbitmq.
Storage service process
CreateDocumentMessage and send the
CreateDocumentReply back to gateway service. Gateway service consumes the reply messages from Rabbitmq and send the response back to client as JSON object. The Rabbitmq consumer implemented with
The communication between publisher and consumer handles with go channels. I have written detailed blog post about go channel based communication and response aggregation in here.
Storage services gets
CreateDocumentMessage from Rabbitmq. The Rabbitmq consumer implemented with
Once message receives, it do what ever the operations(ex: create a document on database) and sends the
CreateDocumentReply back to gateway service via Rabbitmq. The Rabbitmq publisher implemented with
I have dockerized all the services and available to deploy via docker-compose. Following is the
The Rabbitmq connection informations defined with
host.docker.internal field routes to docker host in mac-os. In linux environment change it to local machines IP or add a host entry to
/etc/hosts file by overriding
host.docker.local with local machines IP. Services can be started in following order.
We can send the JSON request to gateway REST api via
curl. Following is the way to test the service. The Rabbitmq admin console can be viewed from