What is “yield” or “generator” in Python?
A lot of new python learners have a hard time wrapping their brain around PEP 380. I am usually asked:
- What does the “yield” keyword do?
- What are the situations where “yield from” is useful?
- What is the classic use case?
- Why is it compared to micro-threads?
To understand what
yielddoes, you must understand what generators are. And before you can understand generators, you must understand iterables.
An iterable is an object that has an
__iter__ method which returns an iterator, or which defines a
__getitem__ method that can take sequential indexes starting from zero (and raises an
IndexError when the indexes are no longer valid). So an iterable is an object that you can get an iterator from.
- anything that can be looped over (i.e. you can loop over a string or file) or
- anything that can appear on the right-side of a for-loop:
for x in iterable: ...or
- anything you can call with
iter()that will return an ITERATOR:
>>> mylist = [1, 2, 3]
>>> # mylist = [x*x for x in range(3)] # or a list comprehension
>>> for i in mylist:
Generators are iterators, a kind of iterable you can only iterate over once. Generators do not store all the values in memory, they generate the values on the fly:
>>> mygenerator = (x*x for x in range(3))
>>> for i in mygenerator:
It is just the same except you used
() instead of
. BUT, you cannot perform
for i in mygenerator a second time since generators can only be used once: they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one.
yield is a keyword that is used like
return, except the function will return a generator.
>>> def func(): # a function with yield
... yield 'I am' # is still a function
... yield 'a generator!'
>>> gen = func()