Photo by Abby Anaday on Unsplash

Intermediate Python Knowledge

Generators in Python — 5 Things to Know

Understand what generators are and how to use them

Yong Cui
Yong Cui
May 4, 2020 · 8 min read

1. What is a generator?

Relationships Between Generators, Iterators, and Iterables
# The general format of the for loop
for item in iterable:
# do something
>>> # A simple example of the for loop
>>> for number in [2, 3, 4]:
... print(f"Number: {number}")
...
Number: 2
Number: 3
Number: 4
>>> # Make an iterator from a string
>>> letter_iterator = iter('ox')
>>> letter_iterator.__next__()
'o'
>>> letter_iterator.__next__()
'x'
>>> letter_iterator.__next__()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> # Define a generator function
>>> def get_number_generator():
... yield 1
... yield 2
...
>>> # Create a generator
>>> number_generator = get_number_generator()
>>>
>>> # Get the elements from the generator one by one
>>> number_generator.__next__()
1
>>> number_generator.__next__()
2
>>> number_generator.__next__()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration

2. What’s the yield keyword?

>>> # Define a function that doesn't explicity return any value
>>> def get_no_returns():
... a = 1
...
>>> returned_value = get_no_returns()
>>> returned_value is None
True
>>> # Define a generator function
>>> def get_number_generator():
... print('Before yielding number 1')
... yield 1
... print('Before yielding number 2')
... yield 2
... print('After yielding both numbers')
...
>>> # Create a generator
>>> number_generator = get_number_generator()
>>>
>>> # Get the elements from the generator one by one
>>> number_generator.__next__()
Before yielding number 1
1
>>> number_generator.__next__()
Before yielding number 2
2
>>> number_generator.__next__()
After yielding both numbers
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration

3. What’s a generator expression?

>>> # Create a list using list comprehension
>>> squared_numbers = [x*x for x in range(10)]
>>> squared_numbers
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> # Generator expressions to create a generator
>>> squared_numbers_gen = (x*x for x in range(10))
>>> type(squared_numbers_gen)
<class 'generator'>
>>>
>>> # Get the elements
>>> squared_numbers_gen.__next__()
0
>>> squared_numbers_gen.__next__()
1

4. Why do we use generators?

>>> number_count = 1_000_000_000
>>>
>>> # Create a list
>>> numbers_list = [x*x for x in range(number_count)]
>>> numbers_list.__sizeof__()
8058558856
>>>
>>> # Create a generator
>>> numbers_gen = (x*x for x in range(number_count))
>>> numbers_gen.__sizeof__()
96

5. What’s a practical example?

>>> import random
>>>
>>> # Create a file with simulated data
>>> out_file = "EEG_data.txt"
>>>
>>> with open(out_file, "w") as eeg_file:
... for _ in range(1000):
... data = [str(random.gauss(0,1)) for _ in range(10000)]
... line_str = " ".join(line_data) + '\n'
... _ = eeg_file.write(line_str)
...
>>>
>>> # Define the needed variables
>>> n = 4
>>> updated_fn = "updated_EEG_data.txt"
>>>
>>> # Use the open() method to create line generator for the files
>>> with open(eeg_fn, "r") as eeg_file, open(updated_fn, "w") as updated:
... # Use the generator in an iteration
... for line_data in eeg_file:
... # Create numbers from the line string data
... numbers = [float(x) for x in line_data.split(" ")]
... calculated_str = list()
... # Calculate means for four-number segments
... for i in range(0, len(numbers), n):
... calculated_str.append(str(sum(numbers[i:i+n])/n))
...
... # Create the line string data for writing
... line_str = " ".join(calculated_str) + '\n'
... # Write the data
... _ = updated.write(line_str)
...
>>>

Before You Go

The Startup

Get smarter at building your thing. Join The Startup’s +788K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Yong Cui

Written by

Yong Cui

Work at the nexus of biomedicine, data science & mobile dev. Love to write on these technological topics. Follow me @ycui01 on Twitter to get latest articles.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +788K followers.

Yong Cui

Written by

Yong Cui

Work at the nexus of biomedicine, data science & mobile dev. Love to write on these technological topics. Follow me @ycui01 on Twitter to get latest articles.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +788K followers.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store