Analytics Vidhya
Published in

Analytics Vidhya

Handling Go routines

StorieCreating a go routine is simple, since go routine is light weight we can create thousands of go routines to execute tasks. But every routine comes with a cost, the cost of consuming resources like memory or CPU usage.

As easy to use go routines is, it is important to manage them as well. We need to make sure that go routines should release compute resources after completion of the tasks assigned. Most of the time we use go routines to execute multiple tasks concurrently, but if we leave go routine hanging, it will lead to a high consumption of compute resources in long run.

The issue of consuming resources and not releasing them once finished will lead to less available resources for other processes to run. This will lead to continuous freezing of system and a drastic load on CPU. The code which was running 10 times fast after using go routines, will start running as slow as 100 times to the previous code. Sometimes only way to resolve this issue is a shutdown.

One way is to assign limited CPU to our golang code using runtime package, and allocate limited resources but this will not resolve the issue. We should implement persistent solution for this problem. Assigning go routines the amount of work and closing them once the task assigned is finished will be the best way to handle it.

Let’s dive into some examples of go routine leak.

Above mentioned code will work as expected with no bugs, but the go routine will keep running due to never ending for loop which causes a leak since there is no signal or way to halt it in case of an error.

Let us look at another example of go routine leak when running multiple go routines which are together writing to the same channel.

Since we are using a default case for select, deadlock will not happen in the routine. Although running the program multiple times, there are chances that values passed to the channel will never received, for example in case of REST api requests.

There is a third party package created to find out if there is a leak inside program by running tests. The package is open source used as go routine leak detector . I have used the same package to find out if go routines running in the program contains a leak.

After creating above test file for the program mentioned in second example. We can the run the test file using go test . It will return the the number of go routine which will fail to complete execution. We need to then verify the point of go routine leak and resolve it.

To resolve above leak we have to make sure all the values sent on a channel should be received. Another option is not to send the values on channel which can not be received.

In above case, I have sent a signal to stop for loop after a certain period of time to make sure that there will be no leakage. There can be other ways to resolve this issue viz. to receive all values sent on channel or not to leave go routine hanging.

Conclusion:- Using go routines is easy, but to manage them is important to make sure that our software will run efficiently in long run. If not handled properly, it will lead to block process and consume large cpu and memory resources to run small chunk of code. Follow for updates….




Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem

Recommended from Medium

The Docker Attack Surface

The Bibliography & Citation module

26 — The Pub Sub, Priority Queue and Pipes and Filter Patterns

Advanced Kubernetes Operator Development with Finalizer, Informer, and Webhook

Google AppEngine & Bazel: How to deploy to appengine flexible using bazel and google cloud deploy ?

Why our ETL tool doesn’t do transformations

Spooked Beans: Baked Beans 2.0?

Percona Series / XtraDB Cluster, 5.7

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Himanshu Chaudhary

Himanshu Chaudhary

Cloud Storage Engineer, IBM

More from Medium

Cell Lifecycle at a given time in Python and Golang

Storing and processing real-time time-series data with Influx DB

Python3 — fixing gunicorn and flask

From Zero to Hero with WebRTC in JavaScript and Python in small snippets of code. Part 4