Testing Python 3 and 2 simultaneously with retox
Python Testing — a background
Python has a built in unit test runner and framework, unittest that got a bit of an overhaul in Python 3. For any modern application, you’re likely trying to test in Python 2 and Python 3, or just Python 3.
With very little effort you can create simple unit tests in Python and execute them using python -m unittest discover test
This is great if your application has no external dependencies. If that’s not the case you want to install some packages using tools like pip from PyPI.
The issue with PIP is that it installs packages into your global package directory by default, so it can be challenging to reproduce your specific application (after all Python is interpreted) environment on other computers.
To isolate the specific versions of the packages you installed, you probably want a virtual environment
Virtual Environments
Virtual environments, are copies of the Python binary with a seperate directory for installed packages. This makes it easier to predictably recreate the same environment on multiple servers or development environments. If your app works great in Django 1.9 but not on 2.0, you want to know!
The challenge with the default virtual environment tool, virtualenv is that it only has one Python version, chosen at creation of the Virtual Environment.
If you need to test your application or library on multiple versions of Python you probably need to either leverage a CI/CD service, like Travis CI or Circle CI.
These tools are great as they give you commit-by-commit test output from your unit tests in multiple scenarios or multiple versions of Python.
However, for private repositories they can be costly. Also, having to push your changes to the git repo every time you want to test, is a bit of an issue.
Using Tox
Tox is a tool that enables you to automate the process of creating a virtual environment just for the purpose of running your test suite. You can configure tox to run against multiple Python interpreters as well as running tools like flake8 and pylint to check your code consistency and style.
The downside to Tox is that the output is very verbose and running tests for large projects can take ages since it runs each environment in sequence. Also, having to toggle back and forth between your IDE/editor and a command line is time consuming and can be frustrating.
If you work on a project that has 1000’s of unit tests (which I do!) and you made a change that broke 100’s of those tests on all versions of Python, your output would be 10000’s of lines long when running tox.
So, I put together Retox.
Using retox instead of calling Tox on the shell
Retox is a simple command line GUI I put together as a wrapper around tox, you call retox and leave it running in a seperate window on your desktop.
Retox will run all of the environments in parallel in a greenlet thread pool, leveraging multiple CPUs or Cores on your hardware. This means the execution of your tests is typically 2–4x faster.
The UI gives step by step feedback for each version, so if your change only broke PyLint, you can see that easily in the UI. Failed environments are marked in red.
Continuous Testing using Watchers
Using the -w flag, you can specify your source path (and possibly your test path) and every time it detects that you have made a change to the files, it will re-run the tests automatically.
Installing retox is easy, just
pip install retox
Using Retox
From any project that has a tox.ini already, simply run
retox
Check out the source code here https://github.com/tonybaloney/retox
