ADVANCED PYTHON PROGRAMMING
Loopin’ Around
This time, we take a look at loops—but do so from a Pythonic point of view, as Guido intended.
Having covered conditional statements, it’s time for the next step in control flow: loops. Again, even though it might sound like a basic topic (and it is), there’s a lot to say about it that’s too nuanced for the first time around, and is worth a revision.
Many Loops
There are many different loops out there, so it’s alright to not be proficient in all of them—but it’s less alright to not be proficient in the ones you use. The most common loop by far is the while
loop, which goes like that:
counter = 0
while counter < 10:
counter += 1
print(counter)
# 1 2 3 4 5 6 7 8 9 10
However, in other languages you have more flavors—like a do-while loop, which happens at least once; or the repeat-until loop, which happens until the condition is false. Older languages would sometimes simulate loops using conditional statements and simple goto commands, like so:
int counter = 0;
LOOP:
counter += 1;
printf("%d\n", counter);
if (counter < 10) {
goto LOOP;
}
Funnily enough, for
loops were just syntactic sugar for while
loops at first—a more compact form for the counter’s boilerplate:
for (int i = 0; i < numbers.length; i++) {
printf("%d\n", numbers[i]);
}
A more interesting version of it would be for-each loops, executing something once for every item in a collection:
for (int number : numbers) {
printf("%d\n", number);
}
Or, in functional languages like Javascript:
numbers.forEach(function(number) {
console.log(number);
});
Python has two loop forms—the classic while
loop, and a for-each loop that is simply called a for
loop; so let’s take a closer look at them!
While Loops
There’s not much to be said about while loops, really. I guess we should mention the break
keyword, which jumps outside the loop:
counter = 0
while counter < 10:
counter += 1
if counter % 7 == 0:
print('BOOM!')
break
print(counter)
# 1 2 3 4 5 6 BOOM!
And the continue
keyword, which skips the rest of the iteration and moves on to the next one:
counter = 0
while counter < 10:
counter += 1
if counter % 2 == 0:
continue
print(counter)
# 1 3 5 7 9
One myth I’d like to bust is that infinite loops are a bad thing. First, because some tasks are inherently infinite, like an echo server:
while True:
data = connection.recv(4096)
if not data:
break
connection.send(data)
Second, because if the loop looks a bit cumbersome, it might be nicer to rephrase with while-true:
message = queue.get()
while message is not None and message != 'end':
print(message)
message = queue.get()# vs:while True:
message = queue.get()
if message is None or message == 'end':
break
print(message)
For the Loop!
For loops are much more interesting—primarily because people coming from other languages often misuse them. So let’s talk about how too loop, Python style! To iterate over a list of colors, a C or Java developer might write this:
>>> colors = ['white', 'black', 'green']
>>> for i in range(len(colors)):
... print(colors[i])
white
black
green
That’s OK, but deals with a lower level abstraction than we’d like: instead “iterate over the colors”, it says “iterate over the numbers from 0 to the length of the colors’ list” and uses it for an index. Why do this, when you can simply:
>>> for color in colors:
... print(color)
white
black
green
gnipooL
One reason to work with indexes is if we’d want to iterate over the colors in reverse; then you can do:
>>> for i in range(len(colors)):
... print(colors[len(colors) - i - 1])# or...>>> for i in range(len(colors) - 1, -1, -1):
... print(colors[i])
But that’s kind of ugly. Why do that, when you can simply:
>>> for color in reversed(colors):
... print(color)
green
black
white
Counting
But surely if we need a counter, it’s better to use—well, a counter. Right?
for i in range(len(colors)):
print(f'{i} {colors[i]}')
But instead, you could simply:
>>> for i, item in enumerate(colors):
... print(f'{i} {color}')
0 white
1 black
2 green
What the built in enumerate
function does is take a sequence, and return a sequence of pairs instead—the first slot being the counter, and the second being the item from the original sequence. You can even set the counter’s initial value:
>>> for i, item in enumerate(colors, 1):
... print(f'{i} {color}')
1 white
2 black
3 green
Looooping
What if we want to iterate over both colors and shapes? We’d have to do this:
>>> shapes = ['triangle', 'square', 'circle']
>>> for i in range(min(len(colors), len(shapes))):
... print(f'{colors[i]} {shapes[i]}')
white triangle
black square
green circle
Or, we could use the built in zip
function, which takes any number of sequences, and returns a sequence of tuples—all the first items of all the sequences, then all the second items of all the sequences, and so on, until one of the sequences drains.
>>> for color, shape in zip(colors, shapes):
... print(f'{color} {shape}')
white triangle
black square
green circle
What if the lists have different lengths? The zip
function stops as soon as one of the lists is over; so we’re back to:
>>> colors = ['white', 'black', 'green', 'blue']
>>> for i in range(max(len(colors), len(shapes))):
... color = colors[i] if i < len(colors) else 'colorless'
... shape = shapes[i] if i < len(shapes) else 'blob'
... print(f'{color} {shape}')
white triangle
black square
green circle
blue blob
In this case, we don’t have a built in function to save us; but we have one that is part of the standard library!
>>> import itertools
>>> for color, shape in itertools.zip_longest(colors, shapes):
... print(f'{color or "colorless"} {shape or "blob"}')
Both the zip
and the zip_longest
functions can do more; but it’s pretty technical and boring, so I’ll add an example and leave it at that.
>>> xs = 1, 2
>>> ys = 3, 4, 5
>>> zs = 6, 7, 8, 9
>>> for x, y, z in zip(xs, ys, zs):
... print(x, y, z)
1 3 6
2 4 7
>>> for x, y, z in itertools.zip_longest(xs, ys, zs):
... print(x, y, z)
1 3 6
2 4 7
None 5 8
None None 9
>>> for x, y, z in itertools.zip_longest(xs, ys, zs, fillvalue=0):
... print(x, y, z)
1 3 6
2 4 7
0 5 8
0 0 9
Or Else
In Python, loops can have an else
clause, too; and it’s a bit confusing at first. After all, isn’t this…
>>> counter = 0
>>> while counter < 3:
... counter += 1
... print(i)
... else:
... print('done')
1
2
3
done
…Pretty much the same as this?
>>> counter = 0
>>> while counter < 3:
... counter += 1
... print(i)
... print('done')
1
2
3
done
The truth is—yes and no. To understand it, let’s talk about the weird choice of keyword first: why else
? Well, loops are not unlike if
statements—you could argue an if
statement is a loop with zero or one iterations, whereas a while
loop keeps checking the condition over and over again. But then, an else
statement is just a piece of code that happens when the condition is not true. In case of an if
statement, it means “instead”; but in a while
loop, it means “eventually”. When the loop ends, and its condition is no longer true, then comes the else
clause’s turn; but isn’t it the same as simply placing the else
clause after the loop?
If the loop ends naturally—if it’s “exhausted”, if you will—then it’s the same. But a loop can end unnaturally, because of a break
statement; and in this case, it’s condition is still true, so the else
statement doesn’t happen. That’s why some people prefer to think of it as a nobreak
clause—it happens if, and only if, the loop ended without breaking.
To see why this can be useful, consider the following scenario: we have a list of user objects, and we’re looking for a user with a specific name.
class User:
def __init__(self, name):
self.name = name
def __repr__(self):
return f'<user: {self.name}>'
Let’s start with an empty list:
>>> users = []
>>> target = 'Dan'
>>> for user in users:
... if user.name == target:
... break
>>> user
Traceback (most recent call last):
...
NameError: name 'user' is not defined
OK, so that’s a bug. We need a default value, in case there are no users:
>>> user = 'nobody'
>>> for user in users:
... if user.name == target:
... break
>>> user
'nobody'
But this doesn’t work when there are some users in the list!
>>> users = [User('Alice'), User('Bob'), User('Charlie')]
>>> user = 'nobody'
>>> for user in users:
... if user.name == target:
... break
>>> user
<user: Charlie>
So it seems we have no choice but to use a flag:
>>> user_found = False
>>> for user in users:
... if user.name == target:
... user_found = True
... break
>>> if not user_found:
... user = 'nobody'
>>> user
'nobody'
This one actually works—but it’s a lot of code for a pretty simple notion: get the next user with this name, or else. How about:
>>> for user in users:
... if user.name == target:
... break
... else:
... user = 'nobody'
>>> user
'nobody'
So it turns out, else
clauses actually make life easier sometimes; but at least from my experience, whenever it does—a function would’ve been even better:
def find_user(users, target):
for user in users:
if user.name == target:
return user
return 'nobody'
else
clauses are also the only way to get out of a nested loop in Python, which doesn’t have labels and goto statements. It can wrinkle your brain, but it works:
>>> xs = 1, 2, 3
>>> ys = 3, 4, 5
>>> for x in xs:
... for y in ys:
... print(x, y)
... if x == y:
... break
... else:
... continue
... break
1 3
1 4
1 5
2 3
2 4
2 5
3 3
In this case, we match all the numbers from xs
with all the numbers from ys
, until the two match—at which point, we break out of the inner loop, but we’re still in the outer one. Luckily, we can hack our way out by detecting the opposite of what we want: whether the inner loop has exhausted naturally. If so, we continue to the next iteration, skipping the last statement—and if not, we reach it and break free. Of course, this would’ve been much simpler:
def print_until_equal(xs, ys):
for x in xs:
for y in ys:
print(x, y)
if x == y:
return
Sentinels
Another interesting built in to mention is the iter
function; it’s used to convert iterable objects, list lists, into iterators—which is what the for
loop works with. You don’t have to call it yourself—it’s called implicitly on the for
loop’s target, kind of like the bool
function is called implicitly on the if
statement’s condition.
However, if you do decide to call it explicitly—it accepts a second, optional argument: sentinel
. If this argument is provided, then the first argument—the iterable, that is—should be a callable function, which will be re-invoked until it produces a value equal to the sentinel. This allows for yet another looping paradigm: keep producing results, until a certain special value is returned to signal game over. For example, if we’d like to read lines from a file until a ‘stop’ line is reached, we could do this:
fp = open(path)
for line in iter(fp.readline, 'stop'):
# Do stuff...
Similarly, if we’d like to read from a socket until it closes, we could do this:
chunks = []
for chunk in iter(sock.recv, b''):
chunks.append(chunk)
data = b''.join(chunks)
But wait! The recv()
takes one argument: bufsize
, the maximum amount of bytes to read. Luckily, a function can easily be turned into a “thunk”—a deferred execution, that is—by wrapping it up in a lambda:
chunks = []
for chunk in iter(lambda: sock.recv(4096), b''):
chunks.append(chunk)
data = b''.join(chunks)
Just a word for the wise: the sentinel
argument is not too common, and if you’re not familiar with this looping paradigm—it looks pretty weird. I think it’s important to know about it, especially if you encounter it in someone else’s code; but it’s like the word “indubitably”—you wouldn’t use it in a sentence yourself.
Conclusion
This time, we talked a lot about loops—and specifically, the two loops available in Python: while
and for
(well, for-each). We saw how to use them properly, and even mentioned little-known features like zip_longest
and sentinels. There was also that strange else
clause, and the inevitable conclusion that functions are just better—so if they’re all that great, why don’t we go and marry one in the next post!
The Advanced Python Programming series includes the following articles:
- A Value by Any Other Name
- To Be, or Not to Be
- Loopin’ Around
- Functions at Last
- To Functions, and Beyond!
- Function Internals 1
- Function Internals 2
- Next Generation
- Objects — Objects Everywhere
- Objects Incarnate
- Meddling with Primal Forces
- Descriptors Aplenty
- Death and Taxes
- Metaphysics
- The Ones that Got Away
- International Trade