Go: How Does Go Stop the World?

Vincent Blanchon
Jan 15 · 4 min read
Illustration created for “A Journey With Go”, made from the original Go Gopher, created by Renee French.

ℹ️ This article is based on Go 1.13.

A “Stop the World” (STW) is a crucial phase in some garbage collector algorithms to get track of the memory. It suspends the execution of the program to scan the memory roots and add write barriers. Let’s review how it works internally and the potential issues it could face.

Stop the World

Stopping the program means stopping the running goroutines. Here is a simple program that will “Stop the World”:

func main() {
runtime.GC()
}

Running the garbage collector will trigger two “Stop the World” phases.

For more information about the garbage collector cycle, I suggest you read my article “Go: How Does the Garbage Collector Mark the Memory?

The first step of this phase is to preempt all running goroutines:

goroutines preemption

Once the goroutines are preempted, they will be stopped at a safe point. Meanwhile, the processors P — running code or in the idle list — will be marked as stopped to not be used to run any code:

P are marked as stopped

Then, the Go scheduler will run and detach each M from their respective P and put them in the idle lists:

Ms are moved to the idle list

Regarding the goroutines running on each M, they will wait in the global queue:

Goroutines are waiting in the global queue

Then, once the world is stopped, the only active goroutine can safely run and start the world when the work is done. The tracing will help to see when this phase happens:

Stop the World “STW” phase in the tracing

System calls

The phase “Stop the World” could impact system calls as well since they could return while the world is stopped. Let’s take an example with a program that intensively does system calls and see how it is handled:

func main() {
var wg sync.WaitGroup
wg.Add(10)

Here is the tracing:

The system call here is exiting while the world is stopped. However, since there is no available P — they are all marked as stopped, as seen in the previous section — the goroutine will be put in the global queue and will run later when the world resumes.

Latencies

The third step of the “Stop the World” involves the M’s to be all detached from their P. However, Go will wait for them to stop voluntarily: when the scheduler runs, during syscall, etc. Waiting for a goroutine to be preempted should be fast, but in some cases, it could lead to some latencies. Let’s take an example that will show an extreme case:

func main() {
var t int
for i := 0;i < 20 ;i++ {
go func() {
for i := 0;i < 1000000000 ;i++ {
t++
}
}()
}

runtime.GC()
}

Here, the “Stop the World” phase takes 2.6 seconds:

A goroutine without function calls will not be preempted, and its P will not be released before the end of the task. That will force the “Stop the World” to wait for it. Several solutions exist to improve preemption in the loops, for more information about it, I suggest you read my article “Go: Goroutine and Preemption.”

A Journey With Go

A Journey With Go Language Programming

Vincent Blanchon

Written by

French Gopher in Dubai

A Journey With Go

A Journey With Go Language Programming

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade