Image for post
Image for post

pytest: Save full error output to file

Alexander Hultnér
Jan 29, 2019 · 2 min read

I recently had a problem with some rare occurring failures in tests, they happened at a rate of about 1 time per 5000 full suite runs. These were property-based tests so input-data was automatically generated.

As to make matters worse the entire test suite froze after the failure before it was unable to print what went wrong, at that point Travis-CI would kill the test-runner before we had a chance to catch it.

The natural next step was to start logging failures to a file while running the test suite continuously on a machine set up for the purpose. I used a snippet from the pytest site. At that point I let the tests run in an infinite loop and the next week I hit my first failure. Turns out I hadn’t looked at the snippet to closely, it only logs which tests failed and no details about which params were used or what went wrong.

So I had to tweak the code some and ended up with the following snippet.

Logs full failures including asserts and fixtures, save as in your pytest root directory

At this point, I started running my tests again and this time I were rather lucky, only about 30 hours later I got my first failure with a proper falsifying, repeatable example.

At this point, we could finally fix the bug. I hope other Python developers using pytest can find this useful to achieve the same results.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store