Using concurrent.futures to Run Code Concurrently

Intuitive Python — by David Muller (25 / 41)

👈 Meet Threads and Processes | TOC | Choosing ThreadPoolExecutor or ProcessPoolExe cutor 👉

The concurrent.futures module provides a unified high-level interface over both Thread and Process objects (so you don’t have to use the low-level interfaces in threading and process).

While concurrent.futures may not be able to run code concurrently in all the ways that you’d expect based on reading the underlying threading and multiprocessing modules, concurrent.futures is a great place to start. It exposes an approachable interface for executing code under threads and processes and gives you the power to run code concurrently without needing to get into the grittiest details of concurrent programming. If you can write a function in Python, you can use concurrent.futures.

The two most important classes provided by concurrent.futures are:

​ concurrent.futures.ThreadPoolExecutor
​ concurrent.futures.ProcessPoolExecutor

ThreadPoolExecutor and ProcessPoolExecutor allow you to run code and retrieve outputs using Thread objects and Process objects, respectively. ThreadPoolExecutor and ProcessPoolExecutor hide the details about the underlying Thread and Process so that you can write clean and clear code. Additionally, since the two classes expose virtually identical interfaces, you can quickly switch strategies between threads and processes…