Parallel Execution with Nose2 in Python

Nose2 is a framework that allows parallel execution out-of-the-box. As with any task for nose2, multiprocessing support is packaged as a plugin. This plugin can be turned on by passing command line options or by updating a config file in the root directory of your project. That said, this plugin should only be used for tests that do not share resources or files across executions because this plugin does not support locking or complex process management.

In this short guide, you will learn how to set up multiprocessing in your test suite and also how to create a custom plugin to leverage the hooks exposed by this framework. The custom plugin will allow you to perform actions at the start and end of the subprocesses created by the multiprocessing plugin.

Set Up

To set up your suite, you will need to create a python project and install pip3. You can do this with a python IDE like PyCharm. Once you have set up the project environment, you will need to create a new text file named requirements.txt in the root of your project. You will add the following content to the requirements.txt file that you just created.

Once this file has been saved with new content, you can go ahead and execute the following command in your terminal to install the above PyPi packages.

This command should talk a few minutes to complete execution and install the packages required for the guide.

Writing a Nose2 Tests

Now we will add the nose tests to our project. Create a directory called test and add an empty file called __init__.py to make it a python package. In this directory, add the python module named test_parameterized_search.py. We will create a simple test case to do a search and use different parameters to create multiple test cases from the same test function. I will briefly describe the code for this file as I expect you to already know how to write a nose2 test in Python. If you want to learn more, you can read my previous blog post about writing Parameterized test with Nose2.

In the code above,

• We have extended an unittest.TestCase. We have added a method called test_search().
• This method takes two arguments, search_engine and xpath. By passing different values for these arguments, we create multiple test cases.
• These arguments are passed in the decorator @params which takes a list of tuples, wherein each tuple is mapping to the function arguments.

If you had read my previous blog post, you will realize that we are no longer passing a path to the chromedriver.exe binary to create a WebDriver. In Selenium 4, initializing the WebDriver object with an executable path is deprecated. Instead, we use the webdriver_manager package which downloads the latest binary or utilizes the most recent cache and helps us abstract this logic from our test case.

Next up, create a INI style config file in the root directory of your project and name it as unittest.cfg. You will add the following contents to this file.

This helps nose2 detect your test cases. You can trigger your test now by executing the nose2 command by simply typing it in your terminal.

You should see the tests running at this point.

Adding Parallel Execution

You can enable the parallel execution plugin by either adding it in the unittest.cfg or using the command line switch.

Adding to Config File

You need to add update the unittest.cfg file and the following contents.

In this config file,

• The plugins is set to the multiprocessing plugin name nose2.plugins.mp to load it in your tests.
• In the multiprocess section, you set always-on to True so that this plugin is enabled.
• Within this plugin section, you set the number of processes to utilize for your test runs using the processes property. It defaults to the number of processors your system has.

If you re-run the nose2 command you should notice that, tests are running in parallel.

Using Command-line Switch

The command line method can’t get much simpler. You can trigger tests in parallel using the following command.

Creating your Custom Plugin

As mentioned earlier, plugins are the building blocks of the nose2 framework. To extend the utility of the Multiprocessing plugin further, you can create a new plugin which will utilize its hooks. In the root directory create another python package called plugins and add an empty __init__.py to it. Create another file called my_custom_plugin.py and add the following contents to it.

The following bullet point will explain the code above,

• We extend the Plugin object exposed by nose2 and gain access to hooks exposed by the multiprocessing plugin.
• The startSubprocess hook, as the name suggests will run right before the test execution allowing you wrap around the test execution and add your custom logic.
• In similar fashion, stopSubprocess runs after the test finishes execution.

There are other hooks available for your plugins and you can read more about them in the hooks reference of the nose2 docs. You can enable this plugin by updating the config file.

Although you can enable multiprocessing quite easily in this manner, there are always overhead costs associated with the creation of an entire process for one test case. Thus, it is generally advised to enable parallel execution using multiprocessing only if your suite size is sufficiently big and can justify the overhead costs associated with process management.

Conclusion

As you can see, enabling multiprocess in the nose2 framework was very simple. If you compare this to the mountain of effort needed to set up the bare-minimum unittest framework, you will start to realize the benefits of a modular framework like nose2.

Lastly, if you have any questions about this blog post, you can reach out to us by joining our Discord server:

Join the Tauk Discord Server!