What is “yield” or “generator” in Python?
What is a “coroutine” in Python?
--
A lot of new python learners have a hard time wrapping their brain around PEP 380. I am usually asked:
- What does the “yield” keyword do?
- What are the situations where “yield from” is useful?
- What is the classic use case?
- Why is it compared to micro-threads?
To understand what
yield
does, you must understand what generators are. And before you can understand generators, you must understand iterables.
Iterables
An iterable is an object that has an __iter__
method which returns an iterator, or which defines a __getitem__
method that can take sequential indexes starting from zero (and raises an IndexError
when the indexes are no longer valid). So an iterable is an object that you can get an iterator from.
- anything that can be looped over (i.e. you can loop over a string or file) or
- anything that can appear on the right-side of a for-loop:
for x in iterable: ...
or - anything you can call with
iter()
that will return an ITERATOR:iter(obj)
>>> mylist = [1, 2, 3]
>>> # mylist = [x*x for x in range(3)] # or a list comprehension
>>> for i in mylist:
... print(i)
1
2
3
Generators
Generators are iterators, a kind of iterable you can only iterate over once. Generators do not store all the values in memory, they generate the values on the fly:
>>> mygenerator = (x*x for x in range(3))
>>> for i in mygenerator:
... print(i)
0
1
4
It is just the same except you used ()
instead of []
. BUT, you cannot perform for i in mygenerator
a second time since generators can only be used once: they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one.
Yield
yield
is a keyword that is used like return
, except the function will return a generator.
>>> def func(): # a function with yield
... yield 'I am' # is still a function
... yield 'a generator!'
...
>>> gen = func()
>>> type(gen)…