The building blocks of asynchronous programming

Robin Siwach

Follow

Aug 2, 2020 · 4 min read

Prerequisites: You should know about iterators and how a for loop in Python works behind the scenes. You don’t need to know anything about generators, coroutines, or the yield keyword.

Just a quick overview of how a for loop works in Python:

OUTPUT :1
2
3
4
5

The code above is equivalent to the code given below:

OUTPUT :1
2
3
4
5

What Are Coroutines?

Coroutines are basically functions whose execution can be paused/suspended at a particular point, and then we can resume the execution from that same point later whenever we want.

We need a mechanism — or to be more precise, a keyword — by which we can insert a checkpoint and tell the program that we want to pause the execution of the function here and return control to the point it called from. We’ll resume the execution whenever we want.

In Python, we can pause the execution of a function using the yield keyword.

So here’s where things get interesting:

• We can think of a coroutine as a function that has one or more checkpoints where the execution will be paused and control will be returned to the point where it was called from.
• In essence, a coroutine is a function divided into many parts, and we can execute each part of a coroutine as we execute each iteration of a for loop using the next function.

Here’s a basic example:

OUTPUT :<class 'generator'>
Function Starts
Function Ends

From the output, we notice a few things:

• First, we need to call the coroutine/function that will give us a generator object.
• That generator object will behave similarly to an iterator, but in the case of an iterator, we are traversing over an iterable. With a generator, we’re executing parts of the coroutine.
• Just as a StopIteration exception is thrown and caught behind the scenes of a for loop, the same happens in this case when the last part of the coroutine is executed.

Now this pausing of the function in between is very interesting and opens up some possibilities:

• When the function is paused, we do nothing, which is the case that we just saw.
• Suppose a variable is being modified several times in a function and we want the value of that particular variable at a certain checkpoint. Then when we pause that function on that particular checkpoint, it returns the value of that variable.

Let’s see an example:

OUTPUT :Function Part 1
5
Function part 2
12
Function part 3

Here, the value of x is returned by yield at different checkpoints, as the function execution has been paused.

Whenever we are executing the last part of the function and there is no yield left in the function, after executing that last part, a StopIteration exception will be raised.

Much like when an iterator tries to execute the next function, but there are no more elements left in the iterable, it also raises the StopIteration exception.

• Suppose we want to send a value (which can be a constant or variable) at a certain checkpoint (i.e. at a certain state of a function). We can also do that using the yield keyword. When we want to send a value, we'll use the send function instead of next.

Let’s see an example:

OUTPUT :Function part 1
6
Function part 2
12
Function part 3

The reason we used next before using send is we can only use send when we are at the yield checkpoint, and yield is on the right side of the expression. So to reach that first yield, we have to use the next function.

Now here comes an interesting application of coroutines. Suppose we want to switch back and forth between two functions like we do in multithreading. In multithreading, until an interrupt is encountered by the OS, it will keep executing. In this case, we can switch whenever we want.

Let’s see an example:

OUTPUT :Function 1 part 1
Function 2 part 1
Function 1 part 2
Function 1 part 3
Function 2 part 2
Function 2 part 3
Function 2 part 4
Function 1 part 4
Function 1 part 5

In this example, we can see that we can switch back and forth between coroutines whenever we want.

So if we write our own custom scheduler that handles the switching between multiple coroutines, we can achieve with single threading what we do with multithreading.

Coroutines have many applications such as concurrency and other programming patterns can also be implemented, like Producer-Consumer or Sender-Receiver in network programming. I’ll be exploring those in upcoming articles.

Coroutines are also the building blocks of many frameworks such as asyncio, twisted, aiohttp. They can also be chained together to make pipelines and solve problems.