Parallel tasks

Launching parallel tasks The concurrent.futures module provides a high-level interface for asynchronously executing callables.

The asynchronous execution can be performed with a separate processes, using ProcessPoolExecutor. It implement the interface defined by the abstract Executor class.

ProcessPoolExecutor

The ProcessPoolExecutor class is an Executor subclass that uses a pool of processes to execute calls asynchronously. ProcessPoolExecutor uses the multiprocessing module, which allows it to side-step the Global Interpreter Lock but also means that only picklable objects can be executed and returned.

An Executor subclass that executes calls asynchronously using a pool of at most max_workers processes. If max_workers is None or not given, it will default to the number of processors on the machine.

When one of the worker processes terminates abruptly, a BrokenProcessPool error is raised.