Staying Safe When Writing Concurrent Code
Intuitive Python — by David Muller (27 / 41)
👈 Choosing ThreadPoolExecutor or ProcessPoolExe cutor | TOC | Building with an Alternate Model: asyncio 👉
In general, it is best to share as little as possible between different threads and processes. Problems tend to start as soon as you try to share objects between different thread or process objects.
Python does include a set of tools for locking and otherwise passing and sharing state when using Thread or Process objects including threading.Lock, threading.Barrier, threading.Semaphore, multiprocessing.Queue, and several others. The aforementioned tools are powerful and can be used to write elegant concurrent programs.
When writing the concurrent code, however, I find it helpful to follow a simple heuristic: share nothing. Even though Python provides powerful tools for writing concurrent programs, they are not the first thing I reach for — instead of using a tool like threading.Lock to coordinate threads, is it possible to create two threads that work without coordinating amongst themselves at all?
At its core, “share nothing” means avoiding writing code that looks similar to this:
def add_elements(a_list):
a_list += ["a", "b", "c"]
return a_list if __name__ == "__main__":
with ThreadPoolExecutor() as executor:
…
