When I started developing Python packages, there was one mistake I did quite often: I forgot to add all dependencies. Additionally, I only tested on my machine for a single Python version.
After reading this article, you will know how to locally and automatically test multiple Python versions in isolated environments. This is a preparation for Continuous Integration tools like Travis. I assume you already know the basics of unit testing in Python and how to package your code.
pyenv is a tool which lets you easily install and switch Python environments on your system. Take a look at the official installation instructions; it’s not a Python package but hooks directly into your shell.
Once it is installed, you can get a list of all available Python environments. In July 2020, there were 427 different versions! I’ve shortened them here to show you the ones I think are interesting:
$ pyenv install --list
Install a Python version:
$ pyenv install 3.8.4
Installed Python-3.8.4 to /home/moose/.pyenv/versions/3.8.4
And use it:
$ pyenv local 3.8.4$ python --version
Python 3.8.4$ pip --version
pip 20.1.1 from /home/moose/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip (python 3.8)
Virtual environment basics
A virtual environment encapsulates the installed packages. Different virtual environments still share the same operating system, the same installed c libraries and executables. The only difference is which packages are available.
You can create a new virtual environment called
venv-tutorial like this:
python -m venv venv-tutorial
It creates a folder which contains all installed packages and few other things. To use it, you need to activate it:
This will add the prefix
(venv-tutorial) in front of your prompt. It will make sure that later calls to
pip use this environment. If you want to stop it again, type
deactivate in the shell. If you delete this folder, the virtual environment is gone.
Testing multiple Python versions without tox
When you claim that your project supports Python 2.7 and 3.5 to 3.8, then you better test those versions. In order to make sure that you install the packages properly, you should create a virtual environment. You might end up with creating a shell script which creates those virtual environments, starts the tests and deletes the virtual environments again.
How to use tox
tox uses a
tox.ini file which is in the package root directory. So your project structure might look like this:
your-awesome-project/ # The git repository
├── setup.py # Dependencies / Package Meta data
├── your_awesome_package/ # Code of the package
│ ├── a_module.py
│ └── another_module.py
├── tests/ # Unit tests
│ ├── test_a_module.py
│ └── test_another_module.py
└── tox.ini # Why you're reading this article
tox.ini file to run the tests in an isolated Python environment for Python 3.6, Python 3.7 and Python 3.8 looks like this:
Note that you need to have the different Python versions already installed. You can do this with pyenv and make them available with the following command:
$ pyenv local 3.8.4 3.7.8 3.6.11
tox within the root folder — the same folder that contains your
The next thing you might want to do is to break a couple of things out of the pytest run. For example, it’s not necessary to run the linter
blackin every single environment. Instead, you can define a
linter environment which is run once:
And finally, you want to run the different environments in parallel for speed:
If one of them fails, you get this type of output:
If you want to run just the linter,
tox -e linter is your friend 🙂
You can also create a matrix of different combinations of Python environments and dependencies you install. Thea Flowers showed this at PyCon 2019 (video), but I never had the need to go down that rabbit hole.
Now… what is nox?
nox is a spin-off of tox. Instead of using a
tox.ini configuration file, it uses a
noxfile.py Python file. It’s pretty similar to tox, but more flexible as it uses Python code:
You can run a single session of nox with
nox -s lint:
Overall, not a huge difference. The output of nox is way nicer and for people who get started, I think the Python code written with nox is a bit simpler to read and understand than the tox.ini configuration file.
In this series, we already had:
- Part 1: The basics of Unit Testing in Python
- Part 2: Patching, Mocks and Dependency Injection
- Part 3: How to test Flask applications with Databases, Templates and Protected Pages
- Part 4: tox and nox
- Part 5: Structuring Unit Tests
- Part 6: CI-Pipelines
- Part 7: Property-based Testing
- Part 8: Mutation Testing
In future articles, I will present:
- Static Code Analysis: Linters, Type Checking, and Code Complexity
Let me know if you’re interested in other topics around testing with Python.