Don’t Read from File Paths; Do This Instead

For most of my development career I was annoyed whenever a library author used a stream or file handle for file input. I was then forced to understand how to create a buffered stream just to read what was in the stinkin’ file! But really, this is the way to do it. Let me explain why.

If You’re the Author of a Library Don’t Assume Users Want to Read a File

“What do you mean?” you ask, “Of course users want to open a JSON file and get its input.”

True, in this one instance, a user might want to read from a file. But in today’s multi-stack world, there are many ways to read a string:

  1. Locally, in a file,
  2. Over the network,
  3. Piped from one command into another,
  4. From a database
  5. Converted from one format into another inter-program.

So then with so many options other than a file, how do you get content from a user?

Read from a Stream Instead

Let’s get more detailed. Here’s a simple library that I’ve created that can sum numbers from several files.

This works great for files. However, let’s say someone wants to use this library, but they’re pulling numbers from a database. And let’s say I’m on vacation in the Bahamas, because all developers have a well-paying job and can afford to take a vacation.

Here’s what the user of the library wants to do:

Take a look at add_numbers_from_db . I found the FileAdder library and wanted to use for the program, but it only takes file paths!

This would be really annoying. If the data is already in the database, you’d have to write to a file, then read that file back in…and so that your server isn’t cluttered with a bunch of files with numbers in them (that you’d have to keep track of with a schema of your own — perhaps based on the numberID or something) you’d need to clean them up.

Meanwhile, I’m enjoying the sunshine and friendly people in the Bahamas, perhaps driving down the beach in a Corvette.

So, instead, let’s change it up a little bit. All it takes is to step into the shoes as a library user.

I will assume the library author has already opened the file. Let’s make 2 modifications to our main program:

(If you’re copying this code, make sure to import io.BufferedReader )

Now, look what we can do. If I’m reading from the database I just have to create an IOString…essentially a file-like object residing in memory. It can be read/written/updated, just like a file, but it only exists until Python does it’s ol’ garbage collection trick.

Now, what about the users that REALLY REALLY want to open with a file. Even though I know how to pass in a stream, sometimes it’s really handy just to pass in the file itself.

Here’s what’s cool…it’s only a simple modification to make this happen:

Or perhaps we figure some of our users are streaming to our library over HTTP. Totally, cool, we’ll just use requests and read from the network stream.

All by using our main stream reading function.

Kinda cool, huh?

Conclusion

The next time you’re writing a library or application that loads data from a file, don’t load it from a file. Load it from a stream or file pointer instead. If you think it would be helpful, provide convenience methods to read form the paths to those sources.

In this article I explained why using a stream is better than using a file path when getting file content.

If you liked this article, please share with a few of your friends. Comment on what you found interesting, frustrating, or bizarre.

❤️❤️❤️

If you’d like to see more articles (and get the articles 2 days early!) then consider becoming a supporter on ko-fi:

https://ko-fi.com/damngood/tiers.

❤️❤️❤️

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store