Photo by mahdis mousavi on Unsplash

A For Loop is a control flow statement for specifying iteration in computer science. In simpler terms, a for loop is a section or block of code which runs repeatedly until a certain condition has been satisfied. It is the most basic concept, without which every programming language will be incomplete.

While exploring the ever-surprising python language, I uncovered that it is an Iterator that plays a huge part in the For Loop and many other higher-order functions such as the range function. This article is written to understand what is an iterator, what is a generator, and how it helps us to understand the crux behind the For Loop.

What is an Iterator in python?

An iterator is any python object which can be iterated upon and returns data one at a time. They are implemented in comprehension, loops, and generators but are not readily seen in plain sight. They are the pillars behind most of the beforementioned functionality of python.

An iterator object must support two methods namely __iter__() and __next__(). This is collectively called an Iterator protocol.

• iter() is a function used to return an iterator by calling the __iter__() method on an Iterable object.
• next() is used to iterate through each element. StopIteration is an exception raised whenever the end is reached.

Wait, iter() method is called on an Iterable object? What is the difference between an Iterator and an Iterable?

Well, an Iterable is an object, that one can iterate over. It generates an Iterator when passed to iter() method. Example of iterable objects includes string, list, tuple, dictionary, etc.

Remember, every iterator is also an iterable, but not every iterable is an iterator.

Now that we have some theoretical knowledge about an iterator let’s see them in action.

Let us first see how the iterable object can be converted into an iterator.

# We know string can be iterated over.
s = "Loreum Ipsum"
for word in s:
    print(word)

Output:

L
o
r
e
u
m
 
I
p
s
u
m

What will happen if we call, next() on the string

s = "Loreum Ipsum"
next(s)

Output:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_41560/3883344095.py in <module>
      1 # Calling next() on string
----> 2 next(s)

TypeError: 'str' object is not an iterator

As we mentioned earlier we first need to call iter method to create an iterator from iterable object then only we can use this next method to access the element one by one. See the code snippet below which corrects that,

s = "Loreum Ipsum"
myIter = iter(s)
print(next(myIter))
print(next(myIter))
print(next(myIter))
print(next(myIter))

Output:

L
o
r
e

Now we will try to create an iterator that will print numbers from 1 to n.

class PrintNumber:
    def __init__(self, max):
        self.max = max

    def __iter__(self):
        self.num = 0
        return self

    def __next__(self):
        if(self.num >= self.max):
            raise StopIteration
        self.num += 1
        return self.num

print_num = PrintNumber(3)

print_num_iter = iter(print_num)
print(next(print_num_iter))  
print(next(print_num_iter))  
print(next(print_num_iter)) 

Output:

1
2
3

Notice, we have called next only 3 times, since we explicitly provided max or n as 3. If we give another call to next it will throw StopIteration.

How does For loop work?

Equipped with our knowledge of iterator and how it needs to be implemented it is very easy now to understand how exactly the for loop works. For Loop calls an iter method to get an iterator and then calls method next over and over until a StopIteration exception is raised, which it handles gracefully to end the loop without breaking the code. That’s all, we have debunk the behind the scene of the For loop.

Still, something feels incomplete, inefficient. To create an iterator we have to implement an Iterator Protocol, a class with iter() and next() method. We also need to keep track of the internal states and handle StopIteration exception so that code is not broken. It seems lots of work. This is where Generator comes in.

Generator in Python,

Simply speaking, a generator is a function which returns an iterator. It is very easy to create a generator in a python. We simply need to define a normal function with one change being, using a keyword yield instead of return. If function contains at least one yield statement then it is a generator.

Difference between return and yield is, while return terminates a function entirely, yield temporarily halts a function while retaining its local values and later continues from there on successive calls.

Below is the generator class which iterates the number from 0 to n and yields a number which is divisible by 7

class divisible_by_7_generator:
    def __init__(self, num):
        self.num = num
    #Generator
    def get_nums_divisible_by_7(self):
        for i in range(0, self.num):
            if (i % 7 == 0):
                yield i
   
n = 100
result = divisible_by_7_generator(n)
print(f"Numbers which are divisible between 0 to {n} are:")
for num in result.get_nums_divisible_by_7():
    print(num, end = ",")

Output:

Numbers which are divisible between 0 to 100 are:
0,7,14,21,28,35,42,49,56,63,70,77,84,91,98,

Python Generator Expression

Simple generator can be easily created on the fly using generator expressions. This are very similar to lambda functions with syntax resembling to the list comprehension in the python, with square backets replaced by round parentheses. Unlike list comprehensions, generator expression follow lazy execution, meaning producing item only when asked for and thus hugely beneficial for memory management.

Check out below example, finding a cube of each element in a list.

myList = [1,2,3,4,5,6]
#list Comprehension
cubeList = [x**3 for x in myList]
# generator expressions are surrounded by parenthesis ()
cubeGenerator = (x**3 for x in myList)

print(cubeList)
print(cubeGenerator)

Output:

[1, 8, 27, 64, 125, 216]
<generator object <genexpr> at 0x0000022F5D3EBD60>

We can use for loop to access the values from the generator as follow,

for i in cubeGenerator: 
    print(i)

Output:

1
8
27
64
125
216

Notice, that list comprehension returns a list while generator returns element one at a time.

Conclusion,

In this article we dug deep into the For Loop and in the process learned about very interesting and useful features such as Iterator, Generator. We also get to understand what is yield and how generator can be implemented.

Finally I would like to list a few frequently ask question related to this topics in interviews:

1. What is difference between an Iterator and an Iterable?
2. Can next() can be called on a string or on an iterable object?
3. What is an Iterator Protocol?
4. What is a Generator? What is difference between an Iterator and a Generator?
5. What is Difference between Generator and a Function?
6. What is Yield and what is difference between Yield an Return?

If you like this article please follow me and don’t forget to like and give feedback. Every feedback will motivate me to explore more and come up with many more interesting topics in future. All the code snippets from the article are available on my GitHub repo if anyone wants to play around. You can get in touch with me on my LinkedIn.