A Not-so-Quick-but-Conceptual guide to Python | Notebook | Intermediate — Part 2

Let me be honest. This notebook took me a lot more time than I excepted because I want my audience to understand the concepts of the some of the topics mentioned below in as granular level as possible, yet not making the notebook lengthy.

As promised, this is the part 2 of Python Intermediate of my Python Tutorial series. In this notebook, I have covered:
1. List Slices and Comprehension
2. Lambda
3. Map, Filter & Reduce
4. Decorator
5. Class
6. Iterable, Iterator and Generator

I have tried to explain the Decorator, and Iterator and Generator with more details. Before diving into this section, I highly recommend you guys to go through these articles if you haven’t.

Python Notebook | Beginners
Python Notebook | Intermediate | Part 1

List Slices

List slices provides an advanced way of retrieving values from a list. Basic list slicing involves indexing a list with two colon-separated integers. These three arguments are lower limit, upper limit and step. This returns a new list containing all the values in the old list between the indices specified. By default, lower limit is at index 0, upper limit is at the last value and step is +1.

You can also take a step backwards. When negative values are used for the first and second values in a slice, they count from the end of list.

The indexing of the iterable item starts from 0 if we take it from left and -1 if we take it from the right.

NOTE: Slicing can also be done on tuple.

squares = [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
print(squares[:])
print(squares[::2])
print(squares[2:8:2])
print(squares[6:])
print(squares[4:14])
print(squares[1:-2])
print(squares[-5:-2])
print(squares[7:1:-2])
print(squares[::-1])
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
[0, 4, 16, 36, 64]
[4, 16, 36]
[36, 49, 64, 81]
[16, 25, 36, 49, 64, 81]
[1, 4, 9, 16, 25, 36, 49]
[25, 36, 49]
[49, 25, 9]
[81, 64, 49, 36, 25, 16, 9, 4, 1, 0]

List Comprehension

List comprehension is a useful way of quickly creating lists using simplified version of for loop statement. A list comprehension can also contain an if statement to enforce a condition on values in the list.

NOTE: Trying to create a list, by any means, in a very extensive range will result in a MemoryError.

var = [2*i for i in range(10**100)]

DON’T EVEN TRY IT!

evens=[i**2 for i in range(10) if i**2 % 2 == 0]
print(evens)
[0, 4, 16, 36, 64]

# modify element of list by index value
num = [1,2,3,4,5,6,7,8]
list_of_index = [i for i in num if num.index(i)%2 == 0]
print(list_of_index)
even_num = [num[i] for i in list_of_index]
print(even_num)
[1, 3, 5, 7]
[2, 4, 6, 8]

Lamda

In Python, anonymous function means a function without a name, whereas we use def keyword to create normal functions. The lambda function is used for creating small, one-time and anonymous function objects in Python. The lambda operator can have any number of arguments, but it can have only one expression. The lambda functions can be assigned to variables, and used like normal functions.

Use lambda functions when an anonymous function is required for a short period of time.

#named function
def polynomial(x):
'''
Function to perform a polynomial calculation having a single variable x
'''
return x**2 + 5*x + 4
print("The result for named function: {}".format(polynomial(-4)))
#lambda
poly = lambda x: x**2 + 5*x + 4
print("The result for anonymous function: {}".format(poly(-4)))
The result for named function: 0
The result for anonymous function: 0

Map, Filter & Reduce

The built-in functions map, filter and reduce are very useful higher-order functions that operate on iterable.

The function map takes a function and an iterable as arguments, and returns a new iterable with the function applied to each argument.

The function filter filters an iterable by removing items that don’t match a predicate (a function that returns ONLY Boolean True).

The function reduce applies a rolling computation to sequential pairs of values in a iterable i.e., wanted to compute the product of a list items, sum of tuple items.

NOTE: Both in map and filter, the result has to be explicitly converted to a list or tuple if you want to print it. Python 2 returns a list by default, but this was changed in Python 3 which returns map or filter object, hence the need for the list or tuple function.

def add_five(x):
return x + 5
num_var = [11, 22, 33, 44, 55]
map_result = list(map(add_five, num_var)) # map
print(map_result)
filter_result = tuple(filter(lambda x: x%2==0, num_var)) # filter
print(filter_result)
from functools import reduce
reduce_result = reduce((lambda x, y: x*y), num_var) # reduce
print(reduce_result)
[16, 27, 38, 49, 60]
(22, 44)
19326120
# what do you think is the output?
mylist = [1,2,3,4,5]
print(tuple(map(lambda x: x if x>2 else 0, mylist)))
print(list(filter(lambda x: x if x>2 else 0, mylist)))
(0, 0, 3, 4, 5)
[3, 4, 5]

Decorator

Decorator are functions which modify the functionality of another function. Let’s go one step at a time to understand decorator. In the beginner’s article, we have mentioned that Python’s function is a first class object which can be

  • dynamically created, destroyed
  • stored in a variable
  • passed to a function as a parameter
  • returned as a value from a function

We have already seen the first point to be valid in the beginner’s article. Let’s validate each of these remaining 3 point.

## functions can be assigned to a variable
def my_pymon(text): 
return "Let's go, {}".format(text.upper())
i_choose_you = my_pymon
print(i_choose_you('pykachu'))
print("="*20)
## functions can be passed as argument to another function
def your_pymon(text):
return f"{text}"
def trainer_select(func):
print(func("I choose you, 'char'mander"))
trainer_select(your_pymon)
print("="*20)
## function returned as a value from another function
def battle_began_with(mons):
def who_won(someone):
return f"In the battle, {someone} won against {mons}"
return who_won
battle = battle_began_with("'char'mander")("pykachu")
print(battle)
Let's go, PYKACHU
====================
I choose you, 'char'mander
====================
In the battle, pykachu won against 'char'mander

When you put the pair of parentheses after the function name in main of code, only then the function gets executed. If you don’t put parentheses after it, then it can be passed around and can be assigned to other variables without executing it.

def my_decor(a_func):
def wrapper_func():
print("I am doing some boring work before executing a_func()")
a_func()
print("I am doing some boring work after executing a_func()")
return wrapper_func
def a_function_requiring_decor():
print("I am the function which needs some decoration!")

print(a_function_requiring_decor())
print("="*50)
a_function_requiring_decor = my_decor(a_function_requiring_decor) #the so-called decorator is happening here
print(a_function_requiring_decor())

The variable a_function_requiring_decor is pointing to the wrapper_func inner function. We are returning wrapper_func as a function when we call my_decor(a_function_requiring_decor). So, decorator wraps a function, modifying its behavior.

Another way to write these decorators is using @ symbol.

def my_decorator(func):
def wrapper():
print("Take the marker and write something on the board.")
func()
print("Well done!")
return wrapper
@my_decorator
def tricky():
print("SOMETHING!")

tricky()
Take the marker and write something on the board.
SOMETHING!
Well done!

Class

Python is an object-oriented programming (OOP) language and objects are created using class which is actually the focal point of OOP. The class describes an object’s blueprint, description or metadata. Multiple object can be instantiated using the same class.

Classes are created using the keyword class and an indented block, which contains class methods.

Let’s take a look at an example.

class Pet:
def __init__(self, genre, name, owner):
self.genre = genre
self.name = name
self.owner = owner

def voice(self): #another method added to the class Pet
print("Growl!")

pokemon = Pet("dog","Arcanine","Tim")
print(pokemon.name)
pokemon.voice()
Arcanine
Growl!

The __ init__ method is the most important method in a class which is called when an instance (object) of the class is created. All methods must have self as their first parameter, although you do not need to pass it explicitly when you call the method.

Within the method definition, self refers to the object itself calling the method. From the above example, we see that

pokemon = Pet("dog","Arcanine","Tim")
print(pokemon.name)
>>> Arcanine
  • When we create the pokemon object from the class Pet, we are passing genre, name and owner as “dog”,”Arcanine”,”Tim” and the object (pokemon) will take the place of self.
  • The attributes are accessed using the dot operator.
  • So pokemon is the object, “Arcanine” is the value of the name attribute of this object.

Hence, we can access the attributes in a class using this way: object.attributes.

Classes can have other methods defined to add functionality to them. These methods are accessed using the same dot syntax as attributes.

NOTE: All methods must have self as their first parameter.

Trying to access an attribute of an instance that isn’t defined causes an AttributeError.

Iterable, Iterator and Generator

Iterable and Iterator

Iteration -> Repetition of a process.

Iterable is a type of object which would generate an Iterator when passed to in-built method iter().

Iterator is an object which is used to iterate over an iterable object using next() method, which returns the next item of the iterable object. Any object that has a next() method is therefore an iterator.

NOTE: List, Tuple, Set, Frozenset, Dictionary are in-built iterable objects. They are iterable containers from which you can get an iterator.

This is what happens.

## Let's see an example
my_tuple = ["apple", "banana", "cherry"]
iterated_tuple = iter(my_tuple)
print(type(iterated_tuple))
print(next(iterated_tuple))
print(next(iterated_tuple))
print(next(iterated_tuple))
<class 'list_iterator'>
apple
banana
cherry
## same thing can be written using for loop
my_tuple = ("apple", "banana", "cherry")
for i in my_tuple:
print(i)
apple
banana
cherry

How for loop actually works?

The for loop can iterate over any iterable.

for element in iterable:
# do something with element

is actually implemented as

# create an iterator object from that iterable
iter_obj = iter(iterable)
# infinite loop
while True:
try:
# get the next item
element = next(iter_obj)
# do something with element
except StopIteration:
# if StopIteration is raised, break from loop
break
  • The for loop creates an iterator object internally, iter_obj by calling iter() on the iterable.
  • Inside the while loop, it calls next() to get the next element and executes further
  • After all the items exhaust, StopIteration exception is raised which is internally caught and the loop ends.

To get a better sense of the internals of an iterator, let’s build an iterator producing the Fibonacci numbers.

from itertools import islice
class fib:
def __init__(self):
self.prev = 0
self.curr = 1

def __iter__(self):
return self

def __next__(self):
value = self.curr
self.curr += self.prev
self.prev = value
return value
f = fib()
print(list(islice(f, 0, 10)))

Generator

A lot of overhead in building an iterator:

  • implement a class with iter() and next() methods
  • raise StopIteration when there was no values to be returned
  • makes the code lengthy

Python Generators are a simple way of creating iterators. All the above mentioned overhead are automatically handled by generators in Python.

Generator is a block of code, same as defining a function, having a yield statement instead of a return statement. If a function contains at least one yield statement (it may contain other yield or return statements), it becomes a generator!

The yield statement suspends function’s execution and sends a value back to caller, but retains enough capability to enable function to resume where it is left off. When resumed, the function continues execution immediately after the last yield run. This allows its code to produce a series of values over time, rather than computing them at once and sending them back like a list. We should use yield when we want to iterate over a sequence, but don’t want to store the entire sequence in memory.

This is how the above Fibonacci number code looks like using generator.

def fib():
prev, curr = 0, 1
while True:
yield curr
prev, curr = curr, prev + curr
f = fib()
print(list(islice(f, 0, 10)))

Let’s see what happened.

Take note that fib is defined as a normal Python function, not as class. However, there’s no return keyword inside the function body. The return value of the function will be a generator.

  1. Now when f = fib() is called, the generator is instantiated and returned. No code will be executed at this point.To be explicit: the line prev, curr = 0, 1 is not executed yet.
  2. Then, this generator instance is wrapped in an islice(). This is itself also an iterator. Again, no code executed.
  3. Now, this iterator is wrapped in a list(), which will take the argument and build a list from it. To do so, it will start calling next() on the islice() instance, which in turn will start calling next() on our f instance.
  4. On the first call, the code prev, curr = 0, 1 gets executed, then the while True loop is entered, and then it encounters the yield curr statement. It will produce the value that’s currently in the curr variable and become idle again. This value is passed to the islice() wrapper, which will produce the value (1 in this case) and list can add the value to the list.
  5. Then, list asks islice() for the next value, which will ask f for the next value, which will unpause f from its previous state, resuming with the statement prev, curr = curr, prev + curr.
  6. Then it re-enters the next iteration of the while True loop, and hits the yield curr statement, returning the next value of curr.
  7. This happens until the output list is 10 elements long. When list() asks islice() for the 11th value, islice() will raise a StopIteration exception, indicating that the end has been reached, and list will return the result: a list of 10 items, containing the first 10 Fibonacci numbers.

There are two types of generators in Python: generator functions and generator expressions. A generator function is any function in which the keyword yield appears in its body. We just saw an example of that. Generator expression is equivalent to list comprehension.

To avoid any confusion between iterable, iterator, generator, generator expression, a {list, set, dict} comprehension, check out this diagram.

As always, I am attaching the notebook below.

I hope you have enjoyed this article. Stay tuned for the Python advanced. Peace. ;)