How To _Avoid_ Parallel Execution of Tests in Go
Because in some circumstances, this can cause random failures and even a loss of hair.
Yesterday, I spent quite a bit of time trying to figure out why a certain number of integration tests (i.e., those that range from the API endpoints down to the database) were failing when running the usual go test ./... command (it’s a bit simplified here, but I’m sure you get the point), whereas they would pass individually or by package with a command such as go test github.com/foo/bar -run TestMe or just go test github.com/foo/bar.
I could see that there were some extra records in the database which were returned by a select * from sometable, but I could not figure out where they came from. Also, as we use GORM for our database abstraction, we have hooks that capture all inserts done in the database during the tests, and then remove those records during the tear down phase of each test, so the database always gets back to a clean state. And guess what? It works so well that even after go test ./... completed with failures, I still couldn’t explain where those unexpected records came from.
At this point, I have to admit I was starting to pull some hair off my head. Figuratively speaking.
I was already suspecting some concurrency somewhere, until I remembered about the -parallel flag to support parallel execution of tests in the go test command:
-parallel n
Allow parallel execution of test functions that call t.Parallel.
The value of this flag is the maximum number of tests to run
simultaneously; by default, it is set to the value of GOMAXPROCS.
Note that -parallel only applies within a single test binary.
The ‘go test’ command may run tests for different packages
in parallel as well, according to the setting of the -p flag
(see ‘go help build’).
Even though we don’t explicitly use the t.Parallel() function in our tests, the last sentence of the description above explained the cause of my troubles: go test was running in tests for different packages in parallel!
So, back to the Makefile, changing the command to go test -p 1 ./... and all tests finally passed 🎉
Kudos to my colleague Matous for finding this solution a few weeks ago while he was facing the same situation on another service of our platform. Too bad I wasted so much time on this problem before remembering about this trick 😬 but on a positive note, this led me to writing this post and I hope it will help you in case you run into the same kind of troubles 😉
