Learning Python: Itertools

Sahil Hashimli
Oct 31, 2020 · 8 min read

Everything you can use “for... in..." on is an iterable; lists, strings, files. Let’s learn more itertools with explanations and examples.

First of all, what is this itertools?

The iterator is defined as object types which contains values that can be accessed or iterated using a loop. There are different iterators that come built-in with Python such as lists, sets, etc. Itertools is the Python module that contains some inbuilt functions for generating sequences using iterators.

There are different types of Iterators:

  1. Infinite Iterators: These type of iterators produce infinite sequences.(count(), cycle(), repeat())
  2. Short sequence iterators (terminating): These iterators produces the sequences which terminate after certain iterations.(accumulate(), chain(), compress(), dropwhile(), filterfalse(), zip_longest() etc.)
  3. Combinatorics generator functions: These generators produce the sequences in combinations related to the input arguments.(product(), permutations(), combinations(), combinations_with_replacement())

count ()

The count () function creates an iterator that returns values that are evenly spaced from the beginning of the number.

itertools.count(start=0, step=1)

When counting with floating point numbers, better accuracy can sometimes be achieved by substituting multiplicative code such as: (start + step * i for i in count()).

Example:

import itertoolsfor i in itertools.count(20, 3):
print(i)
if i > 30:
break

Output:

20
23
26
29
32

In the for loop, we tell the function to start at 20 and step 3 until 30.(If we didn’t do this, this would continue forever but, we have added a break.)

cycle()

itertools.cycle(iterable)

Cycle function make an iterator returning elements from the iterable and saving a copy of each. This function cycles through an iterator endlessly. Repeats indefinitely.

Example:

import itertools# Create a list that shows several places
places = ["home", "school", "cafe", "gym","restaurants", "park"]
places_cycle = itertools.cycle(places)
# If you didn't give a value, this would cycle forever
for i in range(9):
# Then call next() whenever you want another one.
place = next(places_cycle)
# Print this value with format method
print("Going {}".format(place))

Output:

Going home
Going school
Going cafe
Going gym
Going restaurants
Going park
Going home
Going school
Going cafe

repeat()

itertools.repeat(object[, times])

This function will repeat an object over and over again. Unless, there is a times argument.

Example:
for i in itertools.repeat(“Look at this! I'm repeat!”):
print(i)
Output:
Look at this! I'm repeat!
Look at this! I'm repeat!
Look at this! I'm repeat!
Look at this! I'm repeat!
...

If we give times argument:

import itertoolsprint([i for i in itertools.repeat(‘example’, 5)])

Output:

['example', 'example', 'example', 'example', 'example']

accumulate()

itertools.accumulate(iterable[,func,*,initial=None])

This function makes an iterator that returns the results of a function. Functions can be passed around very much like variables. The accumulate() function takes a function as an argument. It also takes an iterable. It returns the accumulated results. The results are themselves contained in an iterable. This may all sound very confusing. I assure you that, when you play with the code it will make sense.

Code

data = [1, 2, 3, 4, 5]result = itertools.accumulate(data, operator.mul)
for each in result:
print(each)

Output

1
2
6
24
120

The operator.mul takes two numbers and multiplies them.

operator.mul(1, 2)
2
operator.mul(2, 3)
6
operator.mul(6, 4)
24
operator.mul(24, 5)
120

In this next example will will use the max function.

Code

data = [5, 2, 6, 4, 5, 9, 1]result = itertools.accumulate(data, max)
for each in result:
print(each)

Output

5
5
6
6
6
9
9

The max function returns the largest item.

5
max(5, 2)
5
max(5, 6)
6
max(6, 4)
6
max(6, 5)
6
max(6, 9)
9
max(9, 1)
9

Passing a function is optional.

Code

data = [5, 2, 6, 4, 5, 9, 1]result = itertools.accumulate(data)
for each in result:
print(each)

Output

5
7
13
17
22
31
32

If no function is specified, the items ​​are summed.

5
5 + 2 = 7
7 + 6 = 13
13 + 4 = 17
17 + 5 = 22
22 + 9 = 31
31 + 1 = 32

chain()

itertools.chain(*iterables)

This function takes a series of iterables and return them as one long iterable.

Example:

import itertools
# from itertools import chain
# a list of odd numbers
odd =[1, 3, 5, 7, 9]
# a list of even numbers
even =[2, 4, 6, 8, 10]
# chaining odd and even numbers
numbers = list(chain(odd, even))
print(numbers)

The odd numbers and even numbers are in separate lists. Chain() function combines them to form a new single list.

Output:

[1, 3, 5, 7, 9, 2, 4, 6, 8, 10]

compress()

itertools.compress(data, selectors)

compress()function makes an iterator that filters elements from data returning only those that have a corresponding element in selectors that evaluates to True.

Example:

import itertools 

prog_lang =['C', 'C++',"C#", 'Java', 'Python',"JavaScript"]
selectors = [False, False, False, True, True, False]

best_programming = itertools.compress(prog_lang, selectors)

for each in best_programming:
print("One of the best programming language is " + each)

In the above code, in the Codes list, we have stored four variables and in the selectors’ list, we have four boolean values. When we use the itertools.compress() then the value False is assigned to ‘C’, False to ‘C++’, False to ‘C#’, False to ‘JavaScript’, True is assigned to ‘Java’ and True to ‘Python’. Now while iterating through the loop we will get the output to which the value True is assigned. So, we get “Java” and “Python” while iterating ‘Best Programming’.

Output:

One of the best programming language is JavaOne of the best programming language is Python

dropwhile()

itertools.dropwhile(predicate, iterable) 

You have to write a function in the predicate section. For example, the def function, lambda etc.

The dropwhile() function of Python returns an iterator only after the func in argument returns false for the first time.

import itertools data = [2, 4, 6, 7, 8, 9, 10, 11, 12]
result = itertools.dropwhile(lambda x: x%2==0, data)
for each in result:
print(each)

Output:

7
8
9
10
11
12

filterfalse()

itertools.filterfalse(predicate, iterable)

This function makes an iterator that filters elements from iterable returning only those for which the predicate is False.

from itertools import filterfalse  

li = [1, 2, 3, 4, 5, 7, 8]

print("Even numbers in the list:")
print (list(itertools.filterfalse(lambda x : x % 2 == 0, li)))
print("If function is a None:")
print (list(itertools.filterfalse(None, li)))

Output:

Even numbers in the list:
[1, 3, 5, 7]
If function is a None:
[]

groupby()

itertools.groupby(iterable, key=None)

Make an iterator that returns consecutive keys and groups from the iterable. The key is a function computing a key value for each element. If not specified or is None, key defaults to an identity function and returns the element unchanged. Generally, the iterable needs to already be sorted on the same key function.

Example:

import itertools 

li = [("a", 1), ("a", 2), ("b", 3), ("b", 4),("c", 5),(None, None)]

key_func_1 = lambda x: x[0]
print("Firt key function:")
for key, group in itertools.groupby(li, key_func_1):
print(str(key) + " :", list(group))
key_func_2 = lambda x: x[1]print("Second key function:")
for key, group in itertools.groupby(li, key_func_2):
print(str(key) + " :", list(group))

Output:

Firt key function:
a : [('a', 1), ('a', 2)]
b : [('b', 3), ('b', 4)]
c : [('c', 5)]
None : [(None, None)]
Second key function:
1 : [('a', 1)]
2 : [('a', 2)]
3 : [('b', 3)]
4 : [('b', 4)]
5 : [('c', 5)]
None : [(None, None)]

islice()

itertools.islice(iterable, start, stop[, step])

This function allows you to cut out a piece of an iterable. Actually, islice() does same as normal list slicing. For example: itertools.islice(range(10),2,5) does the same thing as range(10)[2:5].

Example:

import itertools 

cities = [‘New York’, “Baku”, ‘London’, ‘Istanbul’, ‘Moscow’, ‘Paris’]
few_cities = itertools.islice(cities, 3)
print(list(few_cities))

Output:

['New York', 'Baku', 'London']

starmap()

itertools.starmap(function, iterable)

This function makes an iterator that computes the function using arguments obtained from the iterable.

The difference between map() and starmap() parallels the distinction between function(a,b) and function(*c).

Example:

import operator
from itertools import starmap

li =[(2, 4), (3, 2), (4, 5)]

new_li = list(starmap(operator.mul, li))

print(new_li)

Output:

[8, 6, 20]

takewhile()

itertools.takwwhile(predicate, iterable)

This is kind of the opposite of dropwhile().

import itertools data = [2, 4, 6, 7, 8, 9, 10, 11, 12]
result = itertools.takewhile(lambda x: x%2==0, data)
for each in result:
print(each)

Output:

2
4
6

tee()

itertools.tee(iterable, n=2)

This iterator splits the container into a number of iterators mentioned in the argument. This means that it return n independent iterators from a single iterable.

The default is 2, but you can make as many as needed.

Example:

import itertools 

# using tee() to make a list of iterators
iterator1, iterator2, iterator3 = itertools.tee([1, 2, 3, 4, 5], 3)

# printing the values of iterators
print (list(iterator1))
# if we try to print the same list again, we get an empty list
print (list(iterator1))
print (list(iterator2))
print (list(iterator3))

Output:

[1, 2, 3, 4, 5]
[]
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]

zip_longest()

itertools.zip_longest(*iterables, fillvalue=None)

This function makes an iterator that aggregates elements from each of the iterables. If the iterables are of uneven length, missing values are filled-in with fillvalue. Iteration continues until the longest iterable is exhausted.

The default is None, but you can make as many as needed.

import itertools 

colors = ['red', 'orange', 'yellow', 'green', 'blue']
data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
for each in itertools.zip_longest(colors, data, fillvalue="anything"):
print(each)

Output:

('red', 1)
('orange', 2)
('yellow', 3)
('green', 4)
('blue', 5)
('anything', 6)
('anything', 7)
('anything', 8)
('anything', 9)
('anything', 10)

product()

itertools.product(*iterables, repeat=1)

In the terms of Mathematics Cartesian Product of two sets is defined as the set of all ordered pairs (a, b) where a belongs to A and b belongs to B.

The repeat section’s default is 1, but you can make as many as needed.

import itertools 

num_data = [1, 2]
alpha_data = ['a', 'b', 'c']
result = itertools.product(num_data, alpha_data)
for each in result:
print(each)
print("If we add repeat section:")num_data_2 = [1]
alpha_data_2 = ['a', 'b']
result_2 = itertools.product(num_data_2, alpha_data_2, repeat=2)
for each in result_2:
print(each)

Output:

(1, 'a')
(1, 'b')
(1, 'c')
(2, 'a')
(2, 'b')
(2, 'c')
If we add repeat section:
(1, 'a', 1, 'a')
(1, 'a', 1, 'b')
(1, 'b', 1, 'a')
(1, 'b', 1, 'b')

permutations()

itertools.permutations(iterable, r=None)

Return successive r length permutations of elements in the iterable.

If r is not specified or is None, then r defaults to the length of the iterable and all possible full-length permutations are generated.

import itertools 

data = ['1', '2', '3']
result = itertools.permutations(data)
for each in result:
print(each)
print("If r is specified (ex. r=2):")data_2 = ['1', '2', '3']
result_2 = itertools.permutations(data_2, 2)
for each in result_2:
print(each)

Output:

('1', '2', '3')
('1', '3', '2')
('2', '1', '3')
('2', '3', '1')
('3', '1', '2')
('3', '2', '1')
If r is specified (ex. r=2):
('1', '2')
('1', '3')
('2', '1')
('2', '3')
('3', '1')
('3', '2')

combinations()

itertools.combinations(iterable, r)

Return r length subsequences of elements from the input iterable.

shapes = ['circle', 'triangle', 'square',]
result = itertools.combinations(shapes, 2)for each in result:
print(each)

In this code we make all combos with 2 members.

Output:

('circle', 'triangle')
('circle', 'square')
('triangle', 'square')

Example:

shapes = ['circle', 'triangle', 'square',]
result = itertools.combinations(shapes, 3)for each in result:
print(each)

In this code we make all combos with 3members. It is a bit less exciting.

Output:

('circle', 'triangle', 'square')

combinations_with_replacement()

itertools.combinations_with_replacement(iterable, r)

This one is just like the combinations() function, but this one allows individual elements to be repeated more than once.

shapes = ['circle', 'triangle', 'square',]
result = itertools.combinations_with_replacement(shapes, 2)
for each in result:
print(each)

Output:

('circle', 'circle')
('circle', 'triangle')
('circle', 'square')
('triangle', 'triangle')
('triangle', 'square')
('square', 'square')

Here I tried to talk about itertools in Python. Hope this article is helpful. For more information, you can access Python’s official documentation here.

Star Gazers

“If you want to master something, teach it.”

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store