Go Concurrency Patterns: Dive Deep into Concurrent Programming with Go (Part 1)
Working with Go with a specific architecture like gRPC services, REST frameworks are not enough to utilize a Go application to its maximum, it takes a bit more.
We should dive in to those best practices which are being used from the time of evolution of Go.
Go provides us concurrent programming with the help of goroutines and channels. But, it this just that or something more deep behind that?
To be more precise, we have concurrency patterns to manage resources like an artisan.
In this series of articles, we are going to discover those approaches and patterns that are used widely.
I have listed some popular patterns and their concepts & usage.
Go concurrency Patterns:
- Worker/Thread Pool
- Pipeline
- Job Queue
- Generator
- Semaphore
- Fan-in Fan-out
Basics :
What is a goroutine?
It is a lightweight thread managed by the Go runtime.
Specially, go routines run in the same address spaces, so access to shared memory must be synchronized.
What is a channel?
Channels are a typed conduit through which you can transport values with the channel operator, <-.
By default, sends and receives block until the other side is ready. This allows goroutines to synchronize without explicit locks or condition variables.
Why Select statement is important in go routine concurrency?
The select statement lets a go routine wait on multiple communication operations.
A select blocks until one of its cases can run, then it executes that case. It chooses one at random if multiple are ready.
What sync.Mutex is used for?
We’ve seen how channels are great for communication among goroutines. But what if we don’t need communication? What if we just want to make sure only one go routine can access a variable at a time to avoid conflicts?
This concept is called mutual exclusion, and the conventional name for the data structure that provides it is mutex.
So, Now let’s dive into those listed patterns:
1. Worker/Thread Pool:
Why?
Let’s say you have a a machine with some limited resources and CPU utilization left, a goroutine object only takes 2KB of memory to execute, when you spawn goroutines dynamically, it can drain all your limited resources at once depending on your network traffic and as per the kernel, it will kill the process immediately. So, with limited set of workers and a queue of task coming to them, you can limit the usage of your resources and utilize it’s usage as per your machine permits.
What?
In a simplified way, Worker pool is a pattern to archive concurrency for a limited number of workers to execute large amount of tasks on a task queue.
In our Go ecosystem, we will use goroutines to spawn workers and channel for task queue. Each worker will concurrently pull and execute tasks one by one from queue until the queue is empty. as described on the following illustration above.
To be Continued to Part 2.
References:
