Asynchronous Programming in Python

Asynchronous Programming in Python

Asynchronous programming is a programming paradigm that allows many processes to perform concurrently and independently, without waiting for one to finish before proceeding to the next. In this blog post we will look at what asynchronous programming and how to use it in python.

What is Asynchronous Programming ?

In synchronous programming, the computer executes instructions in a sequential order, blocking execution until a job is completed before going on to the next.
Asynchronous programming, on the other hand, allows the program to begin one operation and then go on to another without stopping execution until the original task is completed.

Asynchronous programming is especially beneficial in circumstances where processes, such as network I/O or database operations, can take a long time to complete. Asynchronous programming can help increase performance and responsiveness in software applications by allowing the program to go on to other activities while waiting for a slow operation to complete.

Depending on the programming language and framework used, asynchronous programming can be accomplished using various strategies such as callbacks, promises, or async/await keywords.

Differences Between Asynchronous Programming and Mutliprocessing

The main difference between asynchronous programming and multiprocessing is that asynchronous programming allows non-blocking operations to occur within a single thread, while multiprocessing uses many cores or processes to complete work simultaneously.

Differences Between Asynchronous Programming and Threading

The main difference between asynchronous programming and threading is that asynchronous programming allows for non blocking operations by using a single thread, while threading uses multiple threads to do multiple tasks in parallel.

For more information about differences: Treading vs Multiprocessing vs Asynchronous Programming

How to use Asyncio ?

First let’s make a simple program which starts to do a job at a time and finishes them.

import time
import asyncio

def do_job(s):
    print(f"{s} started")
    await asyncio.sleep(1)
    print(f"{s} done")

def main():
    start = time.perf_counter()
    todo = ["Job A", "Job B", "Job C"]
    for job in todo:
        do_job(job)
    
    end = time.perf_counter()
    print(f"Took: {end-start:.3f} seconds")

main()

# OUTPUT
# Job A started
# Job Adone
# Job B started
# Job B done
# Job C started
# Job C done
# Took: 3.009 seconds

For using asynchronous programming in Python, we need to use the async await keywords and the asyncio library.

Creating and Running Coroutines

A coroutine is a type of function that can be interrupted and resumed at various times throughout execution. In other words, a coroutine allows you to write code that can be interrupted, do something else, and then resume where it left off.

For creating coroutines, we need to use the async keyword before the def keyword. Calling a coroutine does not call the function that we want to run. It creates a task.

import time
import asyncio

async def do_job(s):
  ...

async def main():
  ...

main()

So, as you see, using main() does not run it. To run it, we need to use asyncio.run(task). We know that main() returns a task that we want to run. So switch main() to:

asyncio.run(main())

Await keyword

if you run this code you probably get a error something like this:

 RuntimeWarning: coroutine '...' was never awaited

If you ever see this error, it probably means you forgot to use this function with the await keyword. Await means wait until this is done and pause the current task. You need to run every coroutine with the await keyword.

import time
import asyncio

async def do_job(s):
    print(f"{s} started")
    await asyncio.sleep(1)
    print(f"{s} done")

async def main():
    start = time.perf_counter()
    todo = ["Job A", "Job B", "Job C"]
    for job in todo:
        await do_job(job)
    
    end = time.perf_counter()
    print(f"Took: {end-start:.3f} seconds")

asyncio.run(main())

# OUTPUT
# Job A started
# Job A done
# Job B started
# Job B done
# Job C started
# Job C done
# Took: 3.009 seconds

As you see, we didn’t get any difference. That’s because we use time.sleep() function, which stops all operations at our code, we need to switch this with asyncio.sleep() which is a coroutine.

await asyncio.sleep(1)

But it didn’t change anything else either. So what is the problem? Remember what the await keyword means. It means to wait until the task is completed. But we don't want to wait for these tasks to finish before starting the next one. We want to start them together and wait until they are finished. We can fix this with task "s" we need to switch for loop with:

tasks = [asyncio.create_task(do_job(item)) for item in todo]
done, pending = await asyncio.wait(tasks)

we use asyncio.create(task) for creating a task from do_job coroutine and we start to run them at the same time.

import time
import asyncio

async def do_job(s):
    print(f"{s} started")
    await asyncio.sleep(1)
    print(f"{s} done")

async def main():
    start = time.perf_counter()
    todo = ["Job A", "Job B", "Job C"]
    
    tasks = [asyncio.create_task(do_job(item)) for item in todo]
    done, pending = await asyncio.wait(tasks)
    
    end = time.perf_counter()
    print(f"Took: {end-start:.3f} seconds")

asyncio.run(main())

# OUTPUT
# Job A started
# Job B started
# Job C started
# Job A done
# Job B done
# Job C done
# Took: 1.009 seconds

The code used here is similar to the one in the mCoding channel’s video.

You can see the video for getting more information about asynchronous programming in python. (Intro to async Python | Writing a Web Crawler)