Mastering File Handling in Python: A Comprehensive Guide Part 10

Published in

Python’s Gurus

5 min readJun 3, 2024

I/O, streams, and requests

I/O stands for input/output, and it broadly refers to the communication between a computer and the outside world. There are several different types of I/O, and it is outside the scope of this chapter to explain all of them, but it’s worth going through a couple of examples. The first one will introduce the io.StringIO class, which is an in-memory stream for text I/O. The second one instead will escape the locality of our computer, and demonstrate how to perform an HTTP request.

Using an in-memory stream

In-memory objects can be useful in a multitude of situations. Memory is much faster than a disk, it’s always available, and for small amounts of data can be the perfect choice.
Let’s see the first example:

# io_examples/string_io.py
import io
stream = io.StringIO()
stream.write('Learning Python Programming.\n')
print('Become a Python ninja!', file=stream)
contents = stream.getvalue()
print(contents)
stream.close()

In the preceding code snippet, we import the io module from the standard library. This is a very interesting module that features many tools related to streams and I/O. One of them is StringIO, which is an in-memory buffer in which we’re going to write two sentences, using two different methods, as we did with files in the first examples of this chapter. We can either call StringIO.write() or we can use print, telling it to direct the data to our stream.

By calling getvalue(), we can get the content of the stream. We then proceed to print it, and finally we close it. The call to close() causes the text buffer to be immediately discarded.

There is a more elegant way to write the previous code:

# io_examples/string_io.py
with io.StringIO() as stream:
  stream.write('Learning Python Programming.\n')
  print('Become a Python ninja!', file=stream)
  contents = stream.getvalue()
  print(contents)

Yes, it is again a context manager. Like the built-in open(), io.StringIO() works well within a context manager block. Notice the similarity with open: in this case too, we don’t need to manually close the stream.

When running the script, the output is:

$ python string_io.py
Learning Python Programming.
Become a Python ninja!

Let’s now proceed with the second example.

Making HTTP requests

In this section, we explore two examples on HTTP requests. We will use
the requests library for these examples, which you can install with pip.

We’re going to perform HTTP requests against the httpbin.org (http://httpbin.org/) API, which, interestingly, was developed by Kenneth Reitz, the creator of the requests library itself.

This library is among the most widely adopted all over the world:

# io_examples/reqs.py
import requests
urls = {
  "get": "https://httpbin.org/get?t=learn+python+programming",
  "headers": "https://httpbin.org/headers",
  "ip": "https://httpbin.org/ip",
  "user-agent": "https://httpbin.org/user-agent",
  "UUID": "https://httpbin.org/uuid",
  "JSON": "https://httpbin.org/json",
}
def get_content(title, url):
  resp = requests.get(url)
  print(f"Response for {title}")
  print(resp.json())
for title, url in urls.items():
  get_content(title, url)
  print("-" * 40)

The preceding snippet should be simple to understand. We declare a dictionary of URLs against which we want to perform HTTP requests. We have encapsulated the code that performs the request into a tiny function, get_content(). As you can see, we perform a GET request (by using requests.get()), and we print the title and the JSON decoded version of the body of the response. Let us spend a few words on this last bit.

When we perform a request to a website, or to an API, we get back a response object, which is, very simply, what was returned by the server we performed the request against. The body of some responses from httpbin.org happens to be JSON encoded, so instead of getting the body as it is (by using resp.text) and manually decoding it calling json.loads() on it, we simply combine the two by leveraging the json() method on the response object. There are plenty of reasons why the requests package has become so widely adopted, and one of them is definitely its ease of use.

Now, when you perform a request in your application, you will want to have a much more robust approach in dealing with errors and so on, but for this chapter, a simple example will do.

Going back to our code, in the end, we run a for loop and get all the URLs. When you run it, you will see the result of each call printed on your console, which should look like this (prettified and trimmed for brevity):

$ python reqs.py
Response for get
{
  "args": {"t": "learn python programming"},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.25.1",
    "X-Amzn-Trace-Id": "Root=1-60a42902-3b6093e26ae375244478",
},
  "origin": "86.8.174.15",
  "url": "https://httpbin.org/get?t=learn+python+programming",
}
... rest of the output omitted ...

Notice that you might get a slightly different output in terms of version numbers and IPs, which is fine. Now, GET is only one of the HTTP verbs, albeit one of the most commonly used. Let us also look at how to use the POST verb. This is the type of request you make when you need to send data to the server. Every time you submit a form on the web, you’re making a POST request. So, let’s try to make one programmatically:

# io_examples/reqs_post.py
import requests
url = 'https://httpbin.org/post'
data = dict(title='Learn Python Programming')
resp = requests.post(url, data=data)
print('Response for POST')
print(resp.json())

The preceding code is very similar to what we saw before, only this time we don’t call get(), but post(), and because we want to send some data, we specify that in the call. The requests library offers much more than this. It is a project that we encourage you to check out and explore, as it’s quite likely you will be using it too.

Running the previous script (and applying some prettifying magic to the output) yields the following:

$ python reqs_post.py
Response for POST
{
  "args": {},
  "data": "",
  "files": {},
  "form": {"title": "Learn Python Programming"},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Content-Length": "30",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.25.1",
    "X-Amzn-Trace-Id": "Root=1-60a43131-5032cdbc14db751fe775",
},
  "json": None,
  "origin": "86.8.174.15",
  "url": "https://httpbin.org/post",
}

Notice how the headers are now different, and we find the data we sent in the form key/value pair of the response body.

We hope these short examples are enough to get you started, especially with requests. The web changes every day, so it’s worth learning the basics and then brushing up every now and then.

That’s it for today. See you tomorrow.

If you love my blogs please consider buying me a book.

Python’s Gurus🚀

Thank you for being a part of the Python’s Gurus community!

Before you go:

Be sure to clap x50 time and follow the writer ️👏️️
Follow us: Newsletter
Do you aspire to become a Guru too? Submit your best article or draft to reach our audience.