Building a Pipeline Architecture in Golang

If you don’t know what is a Pipeline Architecture check out my other text:

https://labs.bawi.io/7-reasons-to-use-pipeline-architecture-93346f604b87

Clone this repo to know all the code:

git clone git@gitlab.com:shodocan/golang-pipeline.git

Let’s say we need to rank students based on age and math grades. But checking grades is a slow operation. Using pipelines will be like this:

The pipeline is composed of a first step that starts the flow and more steps that do the job. Golang makes easier to implement steps. Your step will look like this:

When you run that project you can realize that it’s really slow. but its because we induced 100ms delay when checking grades. but we can always make it parallel and make it faster.

if you have the repository clone please checkout to branch v2

git checkout v2

To allow it to be used to new Steps I created a DefaultStep with a method runInParallel that can be used by any other step. And extended it on my step class allowing it to access that method as if it's his own. The code will look like this:

See that now I say it should have 100 steps running in parallel.

So just by scaling it, we made a script that was used to run in 10 seconds needs only one second to run. That is only the most simple advantage of using a Pipeline Architecture.