Everything You Need To Know About Concurrency In Go

Syed Jafar Naqvi
Dec 9, 2019 · 7 min read

Everyone who has ever heard about Go has also heard about goroutines, we are going to cover all you need to know about executing, utilizing and managing goroutines without breaking your neck. Concurrency is the beauty of Go, in fact, this is what makes Go special. If you master this, you reach the soul of the language and enlightenment would strike you. We are going to cover Goroutines, Channels, WaitGroup and Mutex, this is pretty much all that is required to build amazing concurrent software.

How to make the most of this article

This article will give you a concise explanation and a step by step tutorial of almost every building block in Go’s concurrency model, however, covering everything in detail is not possible over here. So if you feel you need to understand something in detail, just Google the respective headers of each section and you will find a lot of depth. You can bookmark this article and keep it as a reference which you can come back to.

Concurrency Vs Parallelism

If you have not seen Rob Pike’s lecture, please take a look here once you are done with this article. Let us try to understand the difference with use cases for each.

Concurrency: You want to execute 50 database queries at once and the database server is hosted on a separate machine, this is an I/O bound operation and your Go program will not need any CPU after your application has initiated the requests and the queries are being executed by the database’s machine. So, you can launch 50 Goroutines simultaneously, one for each query without draining any resources. Now assume that your application is hosted on a 4 core machine, so in this case, you will not be utilizing a single core completely.

Parallelism: You have to perform some heavy image processing or CSV parsing work and you have 50 images/CSV files, this is a CPU bound operation as each file is going to be processed by your Go application. You can still launch 50 Goroutines simultaneously on your same quad-core processor, but now only 4 tasks would be processed at any given time and all the remaining goroutines will be scheduled on any available CPU one after the other.

Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.

Goroutines

Goroutines are functions or methods that run concurrently with other functions or methods. They can be thought of as lightweight threads but the cost of creating and managing a goroutine is very small as compared to threads and that is why it is normal for an application to have thousands of goroutines running concurrently.

//code001.go
func hello() {
fmt.Println("Hello world goroutine")
}

A new goroutine can be started just by adding the keyword “go” before a function call.

In the above program, we have made the main goroutine sleep for 1 second so that the other goroutine gets enough time, 1 second is just an assumption. What if we could make this more efficient by only waiting till the other goroutine is executing rather than hardcoding the wait time? WaitGroups come to the rescue.

sync.WaitGroup

We can modify the above program by simply replacing the sleep call with a WaitGroup.

//code002.go
func hello(wg *sync.WaitGroup) {
fmt.Println("Hello world goroutine")
wg.Done()
}

Not only is the above program faster, as now we are not wasting any time and the program exits as soon as it is done, but it is more robust as well.

Advantages of Goroutines over Threads

  • Startup: Goroutines have a faster startup time than threads.
  • Cheap: They are only a few kb in stack size and the stack can grow and shrink according to needs of the application whereas in the case of threads the stack size has to be specified and is fixed.
  • A single thread can execute multiple goroutines: Suppose there are 50 Goroutines which are I/O bound. If any Goroutine in that thread blocks for I/O say waiting for user input, then another Goroutine can be executed on that same thread. This avoids the need to create more threads at the OS level and thus saving a lot of time which is wasted in Context Switching. All these are taken care of by the Go Runtime and an I/O bound task is converted to a CPU bound task magically.
  • Channels: Channels are a way by which goroutines communicate with each other and it prevents race conditions from happening when accessing shared memory. Channels can be thought of as a pipe using which Goroutines communicate.

Channels

Channels are the pipes that connect concurrent goroutines. You can send values into channels from one goroutine and receive those values into another goroutine. It is the backbone behind Go’s approach to concurrency:

Do not communicate by sharing the memory; instead, share memory by communicating.

So far we know how to launch goroutines and how to wait for that to finish using sync.WaitGroup, but what if we also want to return something from that goroutine? Let’s say that you have launched a batch job in a separate background goroutine and now you want to know whether the job failed or succeeded. For this tutorial, we will modify the above program to return the string rather than printing it, you can return anything from a goroutine using the same technique. Please go through the below program and read the comments.

//code003.go
func hello(ch chan string) {
ch <- "Hello world goroutine"
}

Sending and receiving from a channel

We can send and receive by placing <- in the appropriate position as explained:

ch <- "Hello world goroutine" // Write to channel
responseFromGoroutine := <-ch // Read from channel

Another very important thing to note here is that sends and receives are blocking. This means, when data is sent to a channel, the control is blocked in the send statement until some other Goroutine reads from that channel. Similarly, when data is read from a channel, the read is blocked until some Goroutine writes data to that channel.

Deadlock

If a Goroutine is sending data on a channel, then it is expected that some other Goroutine should be receiving the data. If this does not happen, then the program will panic at runtime with Deadlock.

//code004.go
package main
func main() {
c:=make(chan string)
c<-"Hello"
}

This will be the output:

fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:

Now, let us try to fix this:

//code005.go
package main

You may want to copy this code and tinker with it so that you get complete clarity with channels and deadlocks.

So far, all the examples are using single capacity channels, Go also supports Buffered Channels which have capacity more than 1. You may not need it right away so just remember that there is something called Buffered Channels and when the need arises, you can read about it here.

Select

This statement is used to choose from multiple send/receive channel operations. It blocks until one of its cases can run, then it executes that case. It chooses one at random if multiple are ready. The syntax is similar to switch except that each of the case statements will be a channel operation.

//code006.go
package main
import (
"fmt"
"time"
)
func task1(ch1 chan string) {
time.Sleep(5 * time.Second)
ch1 <- "Task 1 Complete"
}
func task2(ch2 chan string) {
time.Sleep(2 * time.Second)
ch2 <- "Task 2 Complete"
}

The select statement executes the case which occurs earlier. Task1 completes in 5 seconds and Task2 takes just 2 seconds, so the code will print “Task 2 complete” and exit.

sync.Mutex

Although channels are good enough for all the use cases, it is an overkill for certain specific use cases, like incrementing a global variable from multiple Goroutines.

//code007.go
package main
import (
"fmt"
"sync"
)
var num = 0

Try running the above program multiple times, you will get different output every time instead of the desired output, ie. 500.

You can still solve this problem using channels, but Mutex seems to be a better fit over here.

//code008.go
package main
import (
"fmt"
"sync"
)
var num = 0

Now, 500 is the guaranteed output every single time.

Mutex vs Channels

As mentioned before, you could use channels wherever you can use a mutex. Go’s official documentation states that “A common Go newbie mistake is to over-use channels and goroutines just because it’s possible, and/or because it’s fun.”

To summarise, use channels to distribute tasks to Goroutines and use Mutex when multiple Goroutines need to access or modify a common resource/critical section which in the above case is a global variable.

Final Thoughts

Understanding Go’s concurrency model will make you better in building great Software Systems not just in Go, but also in other programming languages. Once you have understood everything mentioned in this article, you might want to understand how the Go runtime manages Goroutines and how does Goroutines map with Threads at the OS level, why Goroutines are lightweight as compared to threads, etc. I found a very good article answering all those questions here.

The source code samples for all the examples can be found here. Feel free to ask anything in the comments.

Xebia Engineering Blog

The Xebia Engineering Blog

Syed Jafar Naqvi

Written by

Golang/Java/Node.Js

Xebia Engineering Blog

The Xebia Engineering Blog

More From Medium

More from Xebia Engineering Blog

More from Xebia Engineering Blog

More from Xebia Engineering Blog

More from Xebia Engineering Blog

Istio Ingress Request Flow

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade