Python iterators and iterables need not be the same!

I was going through and explaining iterators and generators to a couple of students (in a class at rmotr.com). When we finally got to the iterator protocol part, I realized I was unsure of some technical details. Details that I, and judging by what I found when searching for answers, many others just assumed.

So what are iterators and iterables, and are they distinct? They are distinct. Iterables are classes that implement the __iter__ method, a method which returns an iterator. Iterators are classes that implement the __next__ method (or next in Python 2), which continuously returns the next element until the end.

So this begs the question, does an iterable also have to be an iterator? Or does an iterator also have to be an iterable? Personally I first assumed that answer was no to the first question and yes to the second. It seems many shared this assumption. Actually, even the Python documentation (although it’s just the glossary) says the same thing (you can see for yourself here). Being a scientist I realized something, there was no proof of this fact anywhere! So I decided to look into it.

It wasn’t actually very difficult to test. I used the following code to test.

We just made a very simple iterable and iterator. What do you think happens when this is run? If we go by the earlier assumptions, then this should not work. But it does! We get the small sequence:

1
2
3
4
5

So it goes against our assumptions, and even pydocs itself! In case you are skeptical that maybe the iterator is an iterable for some reason. We can check that easily.

myiterable = Iterable()
myiterator = iter(myiterable)
for e in myiterator:
 print(e)

Which gives us the error:

Traceback (most recent call last):
  File "python", line 23, in <module>
TypeError: 'Iterator' object is not iterable

You can try it out for yourself here. Don’t have to take my word for it.

So there we have it! Iterators do not have to be iterables, or vice versa.

Since generators are closely related to iterators/iterables, you might be curious what they are. Are they iterators? Iterables? Both? Let’s do a quick investigation.

We get true on both statements. Try it out for yourself here. It also works with a for-loop both by calling iter() on it first or without. So we see that generators are both iterables and iterators.

In conclusion. Iterators do not need to be iterables, nor do iterables need to be iterators. Generators are both. Now if this result is more sensible and is more logical, or if the glossary definition (and general assumption) is, is a topic for another time!