Nox: the shining python test automation tool

Photo by Franck V. on Unsplash

Last week I discovered nox a command line tool that automates testing in multiple python environments, similar to tox. It is considered by his creator like the descendant of tox.

Its main advantage over tox for me is that you can take advantage of all the power of the python language in your automation process, but it has others cool features. Also it is used by some great projects like google-cloud-python or urllib3.

Usage

Basics

For the example we are going to imagine a simple project with a module example.py that will have one function adding two numbers.

def addition(a, b):
    return a + b

To test this file, we will also create a test_example.py file.

import pytest

from .example import addition


@pytest.mark.parametrize(('a', 'b', 'c'), [
    (0, 0, 0),
    (1, 2, 3)
])
def test_addition_returns_correct_result(a, b, c):
    assert c == addition(a, b)

Now we will create a noxfile.py to test this file.

import nox


@nox.session
def lint(session):
    session.install('flake8')
    session.run('flake8', 'example.py')


@nox.session
def tests(session):
    session.install('pytest')
    session.run('pytest')

Nox works with the notion of sessions. A session is a callable decorated with nox.session taking a session argument that we use to perform various actions. The two main actions you will usually performed are install and run but there are others.

In the previous example, we defined two sessions, one to check that the module example.py respects the python writing standards with the excellent flake8. Nox will install flake8 package as requested by our first command session.install(‘flake8’) in a new virtual environment created by default in a .nox folder at the top of the project but you can change it. Note that this install command will passed all the arguments it have to the pip install command. The second command runs flake8 with argument example.py. Also note that if you want to run a command like foo -x bar you should pass every string of the command like arguments to the session.run command i.e session.run(‘foo’, ‘-x’, ‘bar’).

The second session checks that our script is working the way we are expecting using the pytest library.

To list all the sessions that are available, you can run this command.

nox -l
Sessions defined in ..
* lint
* tests

If you want to run all the sessions, simply run nox and you will have an output like the following:

nox > Running session lint
nox > Creating virtual environment ..
...
nox > Ran multiple sessions:
nox > * lint: success
nox > * tests: success

Testing against multiple versions of python

You can specify multiple versions of python to test or a specific one using the session decorator.

A single version

@nox.session(python="3.8")
def test(session):
    ...

Multiple versions

@nox.session(python=["3.6", "3.7", "3.8"])
def tests(session):
    ...

if you check the session list again, you’ll see that it’s grown richer.

nox -l
..
* lint
* tests-3.6
* tests-3.7
* tests-3.8
sessions marked with * are selected, sessions marked with - are skipped.

Note that you will need to have the specified python versions installed on your machine for the sessions to run correctly. Also if you don’t want to run all the sessions, you can specify which ones you want through the “-s” option. To check that you will run the sessions you want, you can also combined the “-l” option. For example, giving the previous example, if I want to run the lint and tests-3.7 sessions, I will first check with command:

nox -l -s lint tests-3.7
..
* lint
- tests-3.6
* tests-3.7
- tests-3.8
sessions marked with * are selected, sessions marked with - are skipped.

Note that the selected sessions are preceded by an asterisk and those that will not be runned are preceded by a minus sign.

You can also use the environment variable NOXSESSION to achieve the same goal.

export NOXSESSION=lint,tests-3.7
# you can use set command on Windows
nox -l
...
* lint
- tests-3.6
* tests-3.7
- tests-3.8
sessions marked with * are selected, sessions marked with - are skipped.

Parametrization

Another cool feature of nox is the parametrization of arguments. For example, you can easily create a matrix of tests adapted to your needs. Here we create a matrix of four sessions testing two versions of django with two different databases:

@nox.session
@nox.parametrize('django', ['1.9', '2.0'])
@nox.parametrize('database', ['postgres', 'mysql'])
def tests(session, django, database):
    ...

This will create the following sessions:

nox -l
...
* tests(database='postgres', django='1.9')
* tests(database='mysql', django='1.9')
* tests(database='postgres', django='2.0')
* tests(database='mysql', django='2.0')

Changing nox behaviour in noxfile

You can change some behaviour in the noxfile like the folder where virtual environment will be created or reusing existing virtual environments instead of systematically creating a new one.

nox.options.envdir = ".cache"
nox.options.reuse_existing_virtualenvs = True

Note that arguments passed on the command line have precedence on those defined in the noxfile. For a complete list of options you can use to change the behavior, please refer to this documentation.

Convert tox file to nox file

If you are using tox and you want to change to nox, the latter provides a simple script to create a noxfile from your tox file. You need to install nox with an extra dependency.

pip install --upgrade nox[tox_to_nox]

Then you just need to run the script tox_to_nox in the directory containing your tox file.

tox_to_nox

Note that you will probably need to check the noxfile generating and fix it by hands since the support for tox transformation is minimal by now.

This is all for this tutorial, hope you enjoy it. if you want to know more about nox, I invite you to consult the official documentation.

It is my first tutorial so if you have any comments on how to improve it, I’d love to hear them :)