ADVANCED PYTHON PROGRAMMING

Loopin’ Around

This time, we take a look at loops—but do so from a Pythonic point of view, as Guido intended.

9 min readApr 6, 2020

Having covered conditional statements, it’s time for the next step in control flow: loops. Again, even though it might sound like a basic topic (and it is), there’s a lot to say about it that’s too nuanced for the first time around, and is worth a revision.

Many Loops

There are many different loops out there, so it’s alright to not be proficient in all of them—but it’s less alright to not be proficient in the ones you use. The most common loop by far is the while loop, which goes like that:

counter = 0
while counter < 10:
    counter += 1
    print(counter)
# 1 2 3 4 5 6 7 8 9 10

However, in other languages you have more flavors—like a do-while loop, which happens at least once; or the repeat-until loop, which happens until the condition is false. Older languages would sometimes simulate loops using conditional statements and simple goto commands, like so:

int counter = 0;
LOOP:
counter += 1;
printf("%d\n", counter);
if (counter < 10) {
    goto LOOP;
}

Funnily enough, for loops were just syntactic sugar for while loops at first—a more compact form for the counter’s boilerplate:

for (int i = 0; i < numbers.length; i++) {
    printf("%d\n", numbers[i]);
}

A more interesting version of it would be for-each loops, executing something once for every item in a collection:

for (int number : numbers) {
    printf("%d\n", number);
}

Or, in functional languages like Javascript:

numbers.forEach(function(number) {
    console.log(number);
});

Python has two loop forms—the classic while loop, and a for-each loop that is simply called a for loop; so let’s take a closer look at them!

While Loops

There’s not much to be said about while loops, really. I guess we should mention the break keyword, which jumps outside the loop:

counter = 0
while counter < 10:
    counter += 1
    if counter % 7 == 0:
        print('BOOM!')
        break
    print(counter)
# 1 2 3 4 5 6 BOOM!

And the continue keyword, which skips the rest of the iteration and moves on to the next one:

counter = 0
while counter < 10:
    counter += 1
    if counter % 2 == 0:
        continue
    print(counter)
# 1 3 5 7 9

One myth I’d like to bust is that infinite loops are a bad thing. First, because some tasks are inherently infinite, like an echo server:

while True:
    data = connection.recv(4096)
    if not data:
        break
    connection.send(data)

Second, because if the loop looks a bit cumbersome, it might be nicer to rephrase with while-true:

message = queue.get()
while message is not None and message != 'end':
    print(message)
    message = queue.get()# vs:while True:
    message = queue.get()
    if message is None or message == 'end':
        break
    print(message)

For the Loop!

For loops are much more interesting—primarily because people coming from other languages often misuse them. So let’s talk about how too loop, Python style! To iterate over a list of colors, a C or Java developer might write this:

>>> colors = ['white', 'black', 'green']
>>> for i in range(len(colors)):
...     print(colors[i])
white
black
green

That’s OK, but deals with a lower level abstraction than we’d like: instead “iterate over the colors”, it says “iterate over the numbers from 0 to the length of the colors’ list” and uses it for an index. Why do this, when you can simply:

>>> for color in colors:
...     print(color)
white
black
green

gnipooL

One reason to work with indexes is if we’d want to iterate over the colors in reverse; then you can do:

>>> for i in range(len(colors)):
...     print(colors[len(colors) - i - 1])# or...>>> for i in range(len(colors) - 1, -1, -1):
...     print(colors[i])

But that’s kind of ugly. Why do that, when you can simply:

>>> for color in reversed(colors):
...     print(color)
green
black
white

Counting

But surely if we need a counter, it’s better to use—well, a counter. Right?

for i in range(len(colors)):
    print(f'{i} {colors[i]}')

But instead, you could simply:

>>> for i, item in enumerate(colors):
...     print(f'{i} {color}')
0 white
1 black
2 green

What the built in enumerate function does is take a sequence, and return a sequence of pairs instead—the first slot being the counter, and the second being the item from the original sequence. You can even set the counter’s initial value:

>>> for i, item in enumerate(colors, 1):
...     print(f'{i} {color}')
1 white
2 black
3 green

Looooping

What if we want to iterate over both colors and shapes? We’d have to do this:

>>> shapes = ['triangle', 'square', 'circle']
>>> for i in range(min(len(colors), len(shapes))):
...     print(f'{colors[i]} {shapes[i]}')
white triangle
black square
green circle

Or, we could use the built in zip function, which takes any number of sequences, and returns a sequence of tuples—all the first items of all the sequences, then all the second items of all the sequences, and so on, until one of the sequences drains.

>>> for color, shape in zip(colors, shapes):
...     print(f'{color} {shape}')
white triangle
black square
green circle

What if the lists have different lengths? The zip function stops as soon as one of the lists is over; so we’re back to:

>>> colors = ['white', 'black', 'green', 'blue']
>>> for i in range(max(len(colors), len(shapes))):
...     color = colors[i] if i < len(colors) else 'colorless'
...     shape = shapes[i] if i < len(shapes) else 'blob'
...     print(f'{color} {shape}')
white triangle
black square
green circle
blue blob

In this case, we don’t have a built in function to save us; but we have one that is part of the standard library!

>>> import itertools
>>> for color, shape in itertools.zip_longest(colors, shapes):
...     print(f'{color or "colorless"} {shape or "blob"}')

Both the zip and the zip_longest functions can do more; but it’s pretty technical and boring, so I’ll add an example and leave it at that.

>>> xs = 1, 2
>>> ys = 3, 4, 5
>>> zs = 6, 7, 8, 9
>>> for x, y, z in zip(xs, ys, zs):
...     print(x, y, z)
1 3 6
2 4 7
>>> for x, y, z in itertools.zip_longest(xs, ys, zs):
...     print(x, y, z)
1 3 6
2 4 7
None 5 8
None None 9
>>> for x, y, z in itertools.zip_longest(xs, ys, zs, fillvalue=0):
...     print(x, y, z)
1 3 6
2 4 7
0 5 8
0 0 9

Or Else

In Python, loops can have an else clause, too; and it’s a bit confusing at first. After all, isn’t this…

>>> counter = 0
>>> while counter < 3:
...     counter += 1
...     print(i)
... else:
...     print('done')
1
2
3
done

…Pretty much the same as this?

>>> counter = 0
>>> while counter < 3:
...     counter += 1
...     print(i)
... print('done')
1
2
3
done

The truth is—yes and no. To understand it, let’s talk about the weird choice of keyword first: why else? Well, loops are not unlike if statements—you could argue an if statement is a loop with zero or one iterations, whereas a while loop keeps checking the condition over and over again. But then, an else statement is just a piece of code that happens when the condition is not true. In case of an if statement, it means “instead”; but in a while loop, it means “eventually”. When the loop ends, and its condition is no longer true, then comes the else clause’s turn; but isn’t it the same as simply placing the else clause after the loop?

If the loop ends naturally—if it’s “exhausted”, if you will—then it’s the same. But a loop can end unnaturally, because of a break statement; and in this case, it’s condition is still true, so the else statement doesn’t happen. That’s why some people prefer to think of it as a nobreak clause—it happens if, and only if, the loop ended without breaking.

To see why this can be useful, consider the following scenario: we have a list of user objects, and we’re looking for a user with a specific name.

class User:
    def __init__(self, name):
        self.name = name
    def __repr__(self):
        return f'<user: {self.name}>'

Let’s start with an empty list:

>>> users = []
>>> target = 'Dan'
>>> for user in users:
...     if user.name == target:
...         break
>>> user
Traceback (most recent call last):
  ...
NameError: name 'user' is not defined

OK, so that’s a bug. We need a default value, in case there are no users:

>>> user = 'nobody'
>>> for user in users:
...     if user.name == target:
...         break
>>> user
'nobody'

But this doesn’t work when there are some users in the list!

>>> users = [User('Alice'), User('Bob'), User('Charlie')]
>>> user = 'nobody'
>>> for user in users:
...     if user.name == target:
...         break
>>> user
<user: Charlie>

So it seems we have no choice but to use a flag:

>>> user_found = False
>>> for user in users:
...     if user.name == target:
...         user_found = True
...         break
>>> if not user_found:
...     user = 'nobody'
>>> user
'nobody'

This one actually works—but it’s a lot of code for a pretty simple notion: get the next user with this name, or else. How about:

>>> for user in users:
...     if user.name == target:
...         break
... else:
...     user = 'nobody'
>>> user
'nobody'

So it turns out, else clauses actually make life easier sometimes; but at least from my experience, whenever it does—a function would’ve been even better:

def find_user(users, target):
    for user in users:
        if user.name == target:
            return user
    return 'nobody'

else clauses are also the only way to get out of a nested loop in Python, which doesn’t have labels and goto statements. It can wrinkle your brain, but it works:

>>> xs = 1, 2, 3
>>> ys = 3, 4, 5
>>> for x in xs:
...     for y in ys:
...         print(x, y)
...         if x == y:
...             break
...     else:
...         continue
...     break
1 3
1 4
1 5
2 3
2 4
2 5
3 3

In this case, we match all the numbers from xs with all the numbers from ys, until the two match—at which point, we break out of the inner loop, but we’re still in the outer one. Luckily, we can hack our way out by detecting the opposite of what we want: whether the inner loop has exhausted naturally. If so, we continue to the next iteration, skipping the last statement—and if not, we reach it and break free. Of course, this would’ve been much simpler:

def print_until_equal(xs, ys):
    for x in xs:
        for y in ys:
            print(x, y)
            if x == y:
                return

Sentinels

Another interesting built in to mention is the iter function; it’s used to convert iterable objects, list lists, into iterators—which is what the for loop works with. You don’t have to call it yourself—it’s called implicitly on the for loop’s target, kind of like the bool function is called implicitly on the if statement’s condition.

However, if you do decide to call it explicitly—it accepts a second, optional argument: sentinel. If this argument is provided, then the first argument—the iterable, that is—should be a callable function, which will be re-invoked until it produces a value equal to the sentinel. This allows for yet another looping paradigm: keep producing results, until a certain special value is returned to signal game over. For example, if we’d like to read lines from a file until a ‘stop’ line is reached, we could do this:

fp = open(path)
for line in iter(fp.readline, 'stop'):
    # Do stuff...

Similarly, if we’d like to read from a socket until it closes, we could do this:

chunks = []
for chunk in iter(sock.recv, b''):
    chunks.append(chunk)
data = b''.join(chunks)

But wait! The recv() takes one argument: bufsize, the maximum amount of bytes to read. Luckily, a function can easily be turned into a “thunk”—a deferred execution, that is—by wrapping it up in a lambda:

chunks = []
for chunk in iter(lambda: sock.recv(4096), b''):
    chunks.append(chunk)
data = b''.join(chunks)

Just a word for the wise: the sentinel argument is not too common, and if you’re not familiar with this looping paradigm—it looks pretty weird. I think it’s important to know about it, especially if you encounter it in someone else’s code; but it’s like the word “indubitably”—you wouldn’t use it in a sentence yourself.

Conclusion

This time, we talked a lot about loops—and specifically, the two loops available in Python: while and for (well, for-each). We saw how to use them properly, and even mentioned little-known features like zip_longest and sentinels. There was also that strange else clause, and the inevitable conclusion that functions are just better—so if they’re all that great, why don’t we go and marry one in the next post!