6 Intermediate Python Tips For Better Code

Photo by Rio Lecatompessy on Unsplash

Generator expressions

Generators are the memory-efficient way of dealing with large iterables, data structures, and files in Python. They implement lazy loading as a key feature and are thus employed for most memory-intensive tasks.

Consider a simple example: reading a file object line by line and performing some operation. The naive way to do this in case of both big and small files is by loading the whole thing into memory. Like this:

def read_fully():
    with open('big_file.txt', 'r') as file_object:
        for line in file_object.readlines():
             pass_to_some_function(line)
             # do something else

Obviously, this is a bad idea for any medium sized to large sized files. What we can do instead is make use of generators. Simply replace this with the generator equivalent function with something like:

def read_lines(file_object):
    while True:
        data = file_object.readlines()
        if not data:
            break
        yield data

Now, instead of passing the values one by one to another function, simply call this generator as an iterator in that function:

def pass_to_some_function():
    with open('big_file.txt', 'r') as file_object:
        for line in read_lines(file_object)…