Still Relying on Requests for API Calls? Upgrade to a Superior Alternative Now!

Still using requests ??. Lets talk caveman. “Me use Python to talk to internet. Me start with requests for making web calls. It good for simple stuff, but modern web need more power. HTTPXcome along, like better version of wheel. It handle lots of talking at same time, not slow like old way”.

Imagine navigating the wild world of Python and need to communicate with distant servers using HTTP. This has long been dominated by the requests library because of its simplicity and ease of use. However, as modern web applications and services demand more efficiency, particularly in handling multiple concurrent requests. requests asyncronous nature makes it a no go in this situation.

This is where httpx comes in, a superior alternative. This article explores why httpx is a better choice over requests for modern HTTP handling needs.

Key Advantages of httpx Over requests

1. Asynchronous Capabilities

One of the standout features of httpx is its native support for asynchronous HTTP requests. In contrast, requests operates synchronously, blocking the execution of your program until the HTTP request completes. This can be a big hindrance in applications that need to make multiple HTTP requests or perform other tasks concurrently.

requests (Synchronous):

import requests

response = requests.get("https://api.example.com/data")
print(response.json())

httpx (Asynchronous):

import httpx
import asyncio

async def fetch_data():
    async with httpx.AsyncClient() as client:
        response = await client.get("https://api.example.com/data")
        print(response.json())

asyncio.run(fetch_data())

In the asynchronous example, httpx can handle the request without blocking other operations, allowing for more efficient and responsive applications.

2. Concurrent Request Handling

For applications making multiple HTTP requests, such as fetching data from several APIs or scraping multiple web pages, httpx shines by handling these requests concurrently. This can lead to significant performance improvements compared to requests, which processes each request sequentially.

Example of Concurrent Requests with httpx:

import httpx
import asyncio

async def fetch_all(urls):
    async with httpx.AsyncClient() as client:
        tasks = [client.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        return [response.json() for response in responses]

urls = ["https://api.example.com/data"] * 5
results = asyncio.run(fetch_all(urls))
print(results)

In this example, multiple requests are initiated and processed concurrently, reducing the total execution time significantly.

3. HTTP/2 and HTTP/3 Support

httpx provides built-in support for HTTP/2 and HTTP/3. These newer versions of the HTTP protocol offer performance enhancements such as multiplexing, header compression, and better latency handling. requests only supports HTTP/1.1.

Example of HTTP/2 with httpx:

import httpx

client = httpx.Client(http2=True)
response = client.get("https://http2.example.com")
print(response.text)

Using HTTP/2 can result in faster load times and more efficient use of network resources, which is increasingly important for modern web applications.

4. Automatic Connection Pooling and Keep-Alive

httpx automatically manages connection pooling and keep-alive by default. This means it reuses connections for multiple requests to the same host, reducing the overhead of establishing new connections and improving performance.

Example of Connection Reuse in httpx:

import httpx

client = httpx.Client()
response1 = client.get("https://api.example.com/data1")
response2 = client.get("https://api.example.com/data2")

In contrast, while requests does support connection pooling through the Session object, it's not as seamless or efficient as httpx.

5. Enhanced Testing Capabilities

httpx offers a comprehensive mocking mechanism, making it easier to test HTTP interactions in your applications. This includes features like built-in support for mocking requests and responses directly within your tests.

Example of Testing with httpx Mocking:

import httpx
from httpx import MockTransport

def test_fetch_data():
    def mock_response(request):
        return httpx.Response(200, json={"key": "value"})
    transport = MockTransport(mock_response)
    client = httpx.Client(transport=transport)
    response = client.get("https://api.example.com/data")
    assert response.json() == {"key": "value"}

Mocking allows you to isolate and test your HTTP interactions without making actual network calls, leading to faster and more reliable tests.

Benchmark Example

Here’s a basic comparison using a large number of requests:

import asyncio
import time
import httpx
import requests

# URLs for testing
urls = ["https://api.github.com/users/hordunlarmy"] * 10

# Synchronous with requests
start_time = time.time()
for url in urls:
    response = requests.get(url)
    # print(response.json())
print("Synchronous total time:", time.time() - start_time)


# Asynchronous with httpx
async def fetch_all(urls):
    async with httpx.AsyncClient() as client:
        tasks = [client.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        # for response in responses:
        #    print(response.json())


start_time = time.time()
asyncio.run(fetch_all(urls))
print("Asynchronous total time:", time.time() - start_time)

Key Considerations

• Overhead: Asynchronous processing involves some overhead for setting up the event loop and managing coroutines. For a very small number of requests, this overhead might outweigh the benefits.
• Complexity: Asynchronous code can be more complex to write and debug. If simplicity is a priority and performance is not a major concern, requests might be preferable.

Conclusion

Unlike our caveman days with synchronous requests that could slow down our hunts, httpx shines with asynchronous capabilities, making it perfect for handling many tasks at once without missing a beat. Its streamlined syntax and robust features make it a superior choice for today’s web environments, ensuring your Python code runs as swift and smooth as a well-aimed sling.

Python’s Gurus🚀

Thank you for being a part of the Python’s Gurus community!

Before you go:

• Be sure to clap x50 time and follow the writer ️👏️️
• Follow us: Newsletter
• Do you aspire to become a Guru too? Submit your best article or draft to reach our audience.