Python IO streams in examples
Python IO streams: BytesIO and StringIO in practice.
We’ll be using Python IO streams: BytesIO and StringIO to execute some tasks from the real world: sending a photo to Telegram Bot and get configuration from Netflix Config Server (now it’s a part of Spring Cloud).
We are going to cover 2 different stream classes:
- BytesIO — expects binary-like objects and produces bytes objects
- StringIO — expects and produces str objects
For example, you are writing a Telegram bot that sends your user photos from Unsplash website. The intuitive way to code this task is to save the photo to the disk and then read from that file and send the photo to Telegram, at least, I thought so. But, there is a better way to do it using Python streams.
Python stream way:
Let’s discuss the difference between these 2 approaches. Obviously, the biggest one is that you don’t need to save a file to the disk, this leads to better performance. Using streams is better cause of the speed of using RAM to process bytes-like object, rather than using HDD/SSD to save it on disk and then read it from disk.
Another example is to parse and save the configuration from Java Netflix Config Server. Yeah, this is a very unusual case, but still :) Of course, it could be any text-like data file.
I had such a task at my current job. Our project has a big part written in Java and uses Netflix stack, including Netflix Config Server. All configurations for microservices and different environments are stored there. It provides an ability to get any configuration by requesting a specific link. IMHO, it is a great way to handle different configurations and I am looking for a configuration server written in Python, maybe sometimes I’ll write it by myself.
You can use the function
get_properties_from_config_server giving it an link that your configuration contains and it returns
dict object with all configuration values.
StringIO approach handling configuration file is better cause you don’t need to choose the storage directory. In my example, I am working on MacBook and my microservice is being deployed using Docker. And I used
/tmp folder to store this configuration file, then read from it and deleted this file. But! Another my colleague is using Windows. His OS doesn’t have
/tmp folder. And, in my opinion, it is a bad way to adjust your code to be using folder that suits all the OS, you can just use StringIO to store a config file in RAM.
We have discussed BytesIO and StringIO classes from Python io standard library, have written some code that covers sending a photo to Telegram bot and read a configuration file from Netflix Config Server. Of course, you can use Python streams for your own tasks.
Thanks for the attention to the topic, feel free to leave your questions in the comments for discussion.