Getting Started With Concurrency in Python: Part II — Deadlocks, the Producer-Consumer Model & GIL
“It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair, we had everything before us, we had nothing before us, we were all going direct to Heaven, we were all going direct the other way.” — Charles Dickens, “A Tale of Two Cities”
In this part of the series, we will continue exploring the practical implementation of concurrency in Python. Click/Tap here for Part I, where we discuss the basic implementation of threads and locks in Python.
Deadlocks & All That Jazz
Deadlocks can occur when two (or more) threads are blocking each other’s progress — for example, thread_1 is waiting for the release of lock_2 held by thread_2 upon which it will release its lock — lock_1, but the same is true for thread_2 — it will release lock_2 only upon the release of lock_1 by thread_1.
This deadlock situation is simulated in the code snippet below (which will raise an exception).
Note: We use
daemon threads in this example because the execution of daemon threads is finished/killed when the main thread exits. (Non-daemon threads have to be explicitly killed for the main program to complete.)
One way to solve the deadlock problem is to use timeouts when trying to acquire the locks. If we cannot acquire all locks within the given time frame, we will release all partially acquired locks up to that point, and start over. So, we will do something like the below:
Another solution is obviously a proper explicit management of
.release() by the programmer (which, needless to say, is hard!). We’ll build more on this in Part III of this series.
There’s No Such Thing As Free Lunch (Or Is There?)
Say, we want to process and save thousands of web requests at the same time. Now the task of managing this whole ordeal simply with lots and lots of threads seems to be very dangerous — i.e. not thread-safe (e.g. what if some unintended interaction happens and the shared data is tampered with?). No need to fret — the famous Producer-Consumer model comes to our rescue — and you can find more about how it works here.
In the code below, we use an instance of the Condition object which allows threads to wait for their execution until they receive a notification from the other threads. Think of it as a special kind of lock — in addition to
.release(), it also has such methods as
.wait() method releases the lock and re-acquires it only upon notification from the other thread; the
.notify() method wakes up the threads waiting to be notified. In simpler terms, if
my_queue is full (in this example, we set
capacity to 30 elements), then the producer should wait until it gets note of extra space in the queue from the consumer. If
my_queue is empty, the consumer should wait until it knows that the producer added something to it.
For greater clarity, here is a step-by-step flow:
my_queueis empty, the consumer should wait (we call
.wait()on the condition instance) and loses the lock.
- The producer can now pick the lock up, add something to the queue and notify the consumer of its action by calling
.notify()on the condition instance.
- The consumer wakes up but does not hold the lock yet — the producer must call
.release()on the condition instance, so that the consumer can acquire the lock.
- Now, the consumer can start running again.
my_queueis full, the producer should wait (call
.wait()again) and loses the lock.
- The consumer now picks the lock up, takes something from the queue and notifies the producer of the vacated space by calling
.notify()on the condition instance.
- The producer wakes up and will acquire the lock as soon as the consumer releases it.
- Now, the producer can start running again.
Global Interpreter Lock (GIL)
GIL in CPython allows only one thread to execute at a time, which means this can create bottlenecks in a multi-threaded program.
So, here’s the thing — GIL exists for the following reason:
In Python, all objects have a reference count variable that is a tracker for the number of references pointing to the object. Memory occupied by the object is released only when that variable is zero. This can cause race conditions when different threads are toying with that value at the same time — for example, memory might get incorrectly released when a reference to the object still exists. So, to protect these variables locks were added to all the shared data structures, which (unfortunately) can negatively affect performance for the following reasons:
- there’s a repeated acquisition/release of all the locks, and
- deadlocks are more commonplace when there are lots of locks.
The GIL is a single lock on the interpreter itself which adds a rule that execution of any Python bytecode requires acquiring the interpreter lock. This prevents deadlocks (as there is only one lock) and doesn’t introduce much performance overhead. But it effectively makes any CPU-bound Python program single-threaded.[…] Some languages avoid the requirement of a GIL for thread-safe memory management by using approaches other than reference counting, such as garbage collection.(Source)
GIL is a feature of Python because it’s easy (and not to mention, safe) to implement as you only need to manage one lock. However, it effectively makes your program become single-threaded and increases its execution time (due to all the acquire/release lock overheads).
One way to deal with the bottleneck that GIL can introduce is multi-processing, which we will discuss in Part III of this series.
You can find the Jupyter Notebook used for this article here.
Click Here for Part III: Multiprocessing.