Build A Simple Lambda Architecture data pipeline by using Go and GCP stacks.

Weekend build & Learn — Part 2

Wei Huang
Analytics Vidhya

--

In the last article, we learned about how to build a batch-based product data pipeline by using GCP stacks with Go. If you are interested, please kindly check it out.

For this week, we will learn together how to build a simple lambda architecture data pipeline also with my prefer language Go and the GCP stacks.

What’s the Lambda Architecture?

Lambda architecture is a data-processing architecture designed to handle massive quantities of data by taking advantage of both batch and stream-processing methods.

Three keywords, massive quantities of data (Stream data / Message Queue), Batch and Stream-processing.

Wikipedia — Lambda Architecture Diagram

What’s our use case and Architecture?

--

--