Testing Python 3 and 2 simultaneously with retox

Anthony Shaw
Dec 19, 2017 · 3 min read

Python Testing — a background

Python has a built in unit test runner and framework, unittest that got a bit of an overhaul in Python 3. For any modern application, you’re likely trying to test in Python 2 and Python 3, or just Python 3.

With very little effort you can create simple unit tests in Python and execute them using python -m unittest discover test
This is great if your application has no external dependencies. If that’s not the case you want to install some packages using tools like pip from PyPI.

The issue with PIP is that it installs packages into your global package directory by default, so it can be challenging to reproduce your specific application (after all Python is interpreted) environment on other computers.

To isolate the specific versions of the packages you installed, you probably want a virtual environment

Virtual Environments

Virtual environments, are copies of the Python binary with a seperate directory for installed packages. This makes it easier to predictably recreate the same environment on multiple servers or development environments. If your app works great in Django 1.9 but not on 2.0, you want to know!

The challenge with the default virtual environment tool, virtualenv is that it only has one Python version, chosen at creation of the Virtual Environment.

If you need to test your application or library on multiple versions of Python you probably need to either leverage a CI/CD service, like Travis CI or Circle CI.

These tools are great as they give you commit-by-commit test output from your unit tests in multiple scenarios or multiple versions of Python.

However, for private repositories they can be costly. Also, having to push your changes to the git repo every time you want to test, is a bit of an issue.

Using Tox

Tox is a tool that enables you to automate the process of creating a virtual environment just for the purpose of running your test suite. You can configure tox to run against multiple Python interpreters as well as running tools like flake8 and pylint to check your code consistency and style.

The downside to Tox is that the output is very verbose and running tests for large projects can take ages since it runs each environment in sequence. Also, having to toggle back and forth between your IDE/editor and a command line is time consuming and can be frustrating.

If you work on a project that has 1000’s of unit tests (which I do!) and you made a change that broke 100’s of those tests on all versions of Python, your output would be 10000’s of lines long when running tox.

So, I put together Retox.

Using retox instead of calling Tox on the shell

Retox is a simple command line GUI I put together as a wrapper around tox, you call retox and leave it running in a seperate window on your desktop.

Retox will run all of the environments in parallel in a greenlet thread pool, leveraging multiple CPUs or Cores on your hardware. This means the execution of your tests is typically 2–4x faster.

The UI gives step by step feedback for each version, so if your change only broke PyLint, you can see that easily in the UI. Failed environments are marked in red.

Continuous Testing using Watchers

Using the -w flag, you can specify your source path (and possibly your test path) and every time it detects that you have made a change to the files, it will re-run the tests automatically.

Installing retox is easy, just

pip install retox

Using Retox

From any project that has a tox.ini already, simply run


Check out the source code here https://github.com/tonybaloney/retox

Anthony Shaw

Written by

Group Director of Talent at Dimension Data, father, Christian, Python Software Foundation Fellow, Apache Foundation Member.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade