How to Boost your Python Apps using Httpx and Asynchronous Call
An introduction to Httpx, a Requests like HTTP client based on async and await.
Have you ever tried to scrape many web pages, or request multiple resources from REST-APIs? Yes? If you are like me, you probably used Python and the famous Requests package for doing that. Depending on the number of HTTP-requests and the amount of data per request, we’ve wasted a lot of time waiting until we had all the results. Why? Because we did it sequentially which means we could not fire up a new request before the prior one finished.
Luckily, since Python 3.4 we can execute these requests asynchronously and don’t have to spend so much time with idle waiting. With asynchronous programming, we start a task and while it hasn’t finished, we do some other work. This comes with many benefits, such as improved application performance and enhanced responsiveness.
This programming paradigm is especially well suited for IO-bound tasks like HTTP-requests. For example, we can first fire up a GET request which might take a minute. During that time, we can do something else like firing up another HTTP-request, and another one, and another one…
