Tips to improve speed of your test suite

Appaloosa Store
Jul 27, 2017 · 4 min read

At Appaloosa, most of our tests are written with Rspec, but we still have a certain amount of legacy Test Unit tests. Our Rspec and Test-Unit suites take up to 10 minutes each, so a full test suite runs in around 20 minutes. That’s a lot.

We recently decided to invest time on improving our tests to gain time when developing.

Recurring problems and how to avoid them

build vs build_stubbed vs create

This is why we changed to build or build_stubbed whenever possible.

From the documentation :

build_stubbed is the younger, more hip sibling to build; it instantiates and assigns attributes just like build, but that’s where the similarities end. It makes objects look like they’ve been persisted, creates associations with the build_stubbed strategy (whereas build still uses create), and stubs out a handful of methods that interact with the database and raises if you call them. This leads to much faster tests and reduces your test dependency on a database.

from : https://robots.thoughtbot.com/use-factory-girls-build-stubbed-for-a-faster-test

You should try that approach on any test that does not require a model to actually be inserted in your tests database. In those cases, it’s much faster.

before(:each) vs before(:all)

A before is the best place to setup multiple tests. But when your setup is particularly DB-heavy, with a lot of models being created, you should consider executing it once, for all of your assertions. By default before is before(:each). But careful when using before(:all) because it introduce a global state between your tests.

Avoid unnecessary let!

Use let to define a memoized helper method. The value will be cached across multiple calls in the same example but not across examples.

Note that let is lazy-evaluated: it is not evaluated until the first time the method it defines is invoked. You can use let! to force the method’s invocation before each example.

Sometimes it’s better to put let code into a before(:all) block with all the requirements you need to avoid multiple calls.

Try to remove the bang when refactoring your test, it will avoid forcing invocation before each example.

Parallel tests

We tried using it but our project doesn’t fit easy with parallel_tests. We have lots of random failures (classes not loaded, HTTP calls not stubbed…). We spent a few hours trying to fix all of those failing tests but we stopped because we had always new surprises.

Useless factories associations

When run in a Rails console, you immediately realize the problem:

That’s a lot of time-consuming DB calls. So we started removing these associations, set by hands inside test or turn them into traits:

We did that for most of our factories. This saved around 4–6min on the all test suite. Some factories remain unchanged but it can be tricky when they are used in tens of tests.

We also started using FactoryDoctor. FactoryDoctor is a tool developed by Vladimir Dementyev to inform you when you are creating useless data in tests. It’s not perfect but it will often inform you about useless associations or unneeded create.

Logs

Half of the time is spent in DB tasks, but also 13% in writing logs. So we changed the log level in our config (in config/environment/test.rb). We launched FactoryProf again. Unsurprisingly, Logger::LogDevice#write was not present anymore. With this little tweak, we saved 2 minutes!

Conclusion

What’s interesting is that this task has led the team to a lot of discussions about how we test our code, how and when we rely on database, when and why we should mock…

Further reading :

👋

You should subscribe to our blog to be notified whenever a new article is available!

This article was written by part of Appaloosa’s dev team:
Benoit Tigeot and Christophe Valentin

Want to be part of Appaloosa? Head to Welcome to the jungle.

Appaloosa Store Engineering

The devs behind Appaloosa Store