Cytora Golang Academy: Part one

Learning is at the heart and soul of our engineering culture at Cytora. This passion for learning and growth was echoed throughout my interview process, as well as by my new colleagues when I joined.

I was keen to contribute to this great learning atmosphere as soon as I could, so I am pleased and very excited to share with you the outcome of approximately six weeks of work and preparation: a series of three hour-long magic show themed talks on the topic of Goroutines & Channels.

Welcome to the Cytora Golang Academy!

I’ve created this series of blog posts — a.k.a. the Cytora Golang Academy — to share the content in bite sized portions.

At the end of this series, you will have learned about Goroutines & Channels. But also, I’d like you to feel inspired to create your own content and contribute to the learning culture in your own company — so I’ll be sharing a bonus piece on how to create an engaging tech talk. Double win!

To make the content as relatable and entertaining as possible for my colleagues and you, my readers, I decided to use a magic show theme in all my examples. Each technical concept is presented with a whimsical magic trick application. I hope you have as much fun reading the material as I had creating it! 🎩

Now, let’s get stuck in! A journey of a thousand miles begins with a single step, or in this case… a blogpost!

Easy magic tricks: Concurrency basics in Go

My first session in the series aims to slowly ease you into some concurrency basics in Go, regardless of previous experience.

Concurrency vs parallelism

Understanding the difference between concurrency and parallelism is very important when discussing Goroutines & channels.

But what is concurrency? Put simply, it’s the ability of our application to achieve more than one task. Independently executed processes appear to happen at the same time and allow us to efficiently use computing resources. Concurrent events interleave randomly and can happen in any given order.

With parallelism, however, things really do happen all at once. But this means to be really parallel, you need independent CPUs working at the same time to process possibly related independent computations.

I use the analogy of the concurrent and parallel magician to highlight the difference using a more tangible example.

The magician shuffles the card deck and entertains his audience with jokes using parallelism. He is doing both at the same time.

The magician shuffles the card deck and steals a card using concurrency. At some point, he stopped shuffling and stole a card. But we are not sure the order in which these events happened.

What is a Goroutine?

Next up, Goroutines, which enable us to achieve concurrency. They are lightweight threads that are managed by the Go runtime. They are small, fast and mighty. 💪

Goroutines have some really important use cases. For example:

• Background processing of large files
• Handling user requests in web servers
• Pushing tracking events/logs in the background

Goroutines basics

It’s easy to start — you just need to introduce the go keyword before a function invocation to spin up a Goroutine.

func myFunc() {
    fmt.Println("This is the code I will run concurrently.")
}
func main() {
    go myFunc() //start the goroutine using the go keyword - EASY!
}

The “Goroutine Mentalist” trick starts a magician Goroutine to read the audience’s minds and reveal which playing card they’re thinking of! 🤯

When we run this example, it doesn’t work. The main Goroutine started the magic show, started the magician Goroutine, then main had no more work to do and closed down the magic show. Once the main Goroutine shuts down, all its Goroutines are also killed.

The problem is that our magician Goroutine never had the chance to return its results before the main Goroutine shut down the whole program!

We need to remember that when a new Goroutine is started, the Goroutine call returns immediately. It is non-blocking by design so that it allows the Goroutine that it is created from to continue its work. If the main Goroutine terminates then the program will be terminated and no other Goroutine will run.

However, if we give our magician time to do his trick, it’s possible to do this successfully. We can force the main Goroutine to wait for the magician Goroutine to complete its work before shutting down. We can achieve this by adding a sufficiently long sleep that gives the Goroutine time to complete and report back to the main Goroutine.

This reloaded trick makes use of a very long (and resource wasteful!) sleep to get the main Goroutine to wait for the magician Goroutine. As such, this is only an introductory solution that highlights the need for better synchronisation mechanisms.

Please note that this sleep solution was only used here for demonstration purposes, so never use something like this in production grade systems.

The sync package

The Go sync package is a natural next step for discussion.

This gives us synchronisation primitives and locks. Today, we’re going to focus on the WaitGroup and the Mutex from the sync package.

The “Bullet Catch” trick starts up two Goroutines: an assistant and a magician. The assistant shoots the magician, who then catches the bullet with incredible speed and is completely unharmed! 👏

The version of the trick highlights the random scheduling of Goroutines by the Go runtime. As we can see from the output, the magician makes the career ending mistake of revealing the caught bullet before the assistant performs the shot. What a disaster!

Remember that the Go runtime is the one that decides the order in which Goroutines execute. Even though we start the “assistant” Goroutine before the “magician” Goroutine, they can be executed in any order. The Go runtime is the boss of the Goroutines, not us!

The second version of the trick uses the sync.WaitGroup to ensure the completion of Goroutines in a specific order, instead of the random order that the Go runtime executes them in.

The sync.WaitGroup is really intuitive to understand and use. It:

• Has three functions: Add(int), Wait() and Done()
• Has an inner counter that starts at zero and maintains the state of the WaitGroup
• Adds n to the counter when you call Add(n)
• Removes 1 from the counter when you call Done()
• Wait() blocks the current Goroutine and it will be released when the counter is zero
• Should always be passed by reference/pointer to Goroutines

The sync.WaitGroup also removes the rudimentary sleep solution from the previous section which was our inefficient first solution, while ensuring that the Goroutines execute in their intended order.

Next up, the “Infinite Hat” trick starts several Goroutines — our “magicians” — which each place an item in a magic hat. The contents of the impossibly large hat are revealed after the trick! 🎩

The trick makes use of the sync.WaitGroup to wait for all the magicians to place their items. In the reveal, we see that some items placed in the magic hat go missing. This is an unintended consequence of several Goroutines modifying a shared resource.

When a program runs concurrently, the parts of code which modify shared resources, known as the critical section, should not be accessed by multiple Goroutines at the same time.

The second iteration of the trick demonstrates the use of the sync.Mutex to construct a threadsafe magic hat struct that will not be prone to data races when written to by multiple Goroutines.

Just like the sync.WaitGroup, the sync.Mutex is very easy to use:

• The sync.Mutex has 2 methods: Lock() and Unlock()
• Any code that is present between a call to Lock and Unlock will be executed by only one Goroutine. This prevents intermittent bugs and inconsistent results
• If one Goroutine already holds the lock and if a new Goroutine is trying to acquire a lock, the new Goroutine will be blocked until the mutex is unlocked
• Should always be passed by reference/pointer to Goroutines

Using a Mutex, the hat will only be used by one magician Goroutine at a time. As placing items in the hat was the critical section of the trick, this is the part operation that will be protected by the Mutex.

type MutexInfiniteHat struct {
    m *sync.Mutex
    contents string
}

func (h *MutexInfiniteHat) placeItem(s string) {
    h.m.Lock() //acquire control of the hat
    h.contents = fmt.Sprintf("%s\n%s", h.contents, s)
    h.m.Unlock() //relinquish control of the hat
}

Finally, the reveal becomes stable and shows exactly all the items placed inside the hat during the trick. No more missing items!

Channels basics

Channels are pipes through which Goroutines communicate, and there are a few key points to be aware of:

• Each channel has a type it is allowed to transport associated with it.
• chan T is a channel of type T. No other type than T is allowed to be transported using the channel.
• The zero value of a channel is nil.
• Channels are defined using make similar to maps and slices

a := make(chan T)

• Channels can be closed to signal that no other values will sent through the pipe

close(a)

Data travels through channels via send and receive operations:

• The syntax to receive from a channel a is

data := <- a

• The arrow points outwards from a and hence we are reading from channel a and storing the value to the variable data.
• The syntax to write to a channel a is

a <- data

• The arrow points towards a and hence we are writing data to channel a.
• Buffered channels can be created by passing an additional capacity parameter to the make function which specifies the size of the buffer. Buffered channels can hold multiple values, after which they are considered full.

a := make(chan T, capacity)

I used the “Quick Change” trick to demonstrate sending and receiving to channels. The trick starts a “magician” Goroutine, who then very quickly changes his outfit behind a curtain! 🕺🏻

This trick demonstrates how a Goroutine can use a channel to signal completion to the main Goroutine, as the operations of sending and receiving on channels are blocking. This means that the Goroutine attempting to perform the send or receive will block until it can successfully perform its operation. The main Goroutine will wait for the magician Goroutine to do its work, then write to its channel that it has finished before terminating. This example could have also been done using a sync.WaitGroup for synchronisation instead of a channel. See the same “Quick Change” trick performed using a sync.WaitGroup.

Curtain Drop/Conclusions

This brings us to our conclusions and curtain drop. The magic tricks in this session showcase the basic concepts of Goroutines, synchronisation and channels.

The full list of magic tricks presented is:

• Mentalist trick: doing basic work with Goroutines and waiting for them to complete with sleep
• Bullet Catch trick: guarantee Goroutine execution completion and order using the sync.WaitGroup
• Infinite Hat trick: ensure Goroutines acquire orderly control of shared resources
• Quick Change trick: signalling work is completed using channels instead of sync.WaitGroup and demonstrate the blocking behaviour of channel sends/receives

Talk materials

My slide deck was designed to be well documented and self standing.

You can find it here. It fully elaborates on the synopsis I have written in this blogpost.

Keep reading for more magic tricks in part two!