Python stories, October 2018

I’m running @pythonetc, a Telegram channel about Python and programming in general. Here are the best posts of October 2018.

The -- separator

Every well-behaved command line utility should accept arguments in the form of options (e. g. -h or --help), options with parameters (--log-level 2) or positional parameters (cp file1 file2).

Options are distinguished from positional parameters by having a leading dash (or two dashes). Problems start when positional arguments have to start with a dash, e. g. you want to remove a file with the name of -rf: rm -rf doesn't work this way.

The conventional way to solve this problem is to support -- delimiter. Arguments after -- are never interpreted as options:

$ echo test > -rf
$ cat -rf
cat: invalid option -- 'r'
Try 'cat --help' for more information.
$ cat -- -rf
test
$ rm -- -rf
$ cat -- -rf
cat: -rf: No such file or directory

The argparse module automatically handles -- for you.

Stable sot

The default Python sorting is stable meaning it preserves the order of equal objects:

In : a = [2, -1, 0, 1, -2]
In : sorted(a, key=lambda x: x**2)
Out: [0, -1, 1, 2, -2]

max and min functions also tries to be consistent with sorted. max works as sorted(a, reverse=True)[0] while min does sorted(a)[0]. That means that both max and min returns the leftmost possible answer:

In : max([2, -2], key=lambda x: x**2)
Out: 2
In : max([-2, 2], key=lambda x: x**2)
Out: -2
In : min([2, -2], key=lambda x: x**2)
Out: 2
In : min([-2, 2], key=lambda x: x**2)
Out: -2

Default argument cache

Probably the most common newbie mistake with Python it providing a mutable object as a default function argument. That object is shared between all function calls that can lead to bizarre results:

def append_length(lst=[]):
    lst.append(len(lst))
    return lst
print(append_length([1, 2])) # [1, 2, 2]
print(append_length())       # [0]
print(append_length())       # [0, 1]

However, for various caches sharing may be a good thing:

def fact(x, cache={0: 1}):
    if x not in cache:
        cache[x] = x * fact(x - 1)
    return cache[x]
print(fact(5))

In this example, we store calculated factorial values inside the default function value. It can even be extracted:

>>> fact.__defaults__
({0: 1, 1: 1, 2: 2, 3: 6, 4: 24, 5: 120},)

pathlib

Python allows you to work with filesystem paths with the os.path module. The module contains a lot of functions that treat strings as paths and perform useful operations like concating paths and stuff like that:

>>> import os.path
>>> os.path.join('/usr', 'local')
'/usr/local'
>>> os.path.dirname('/var/log')
'/var'

However, since Python 3.4 the new pathlib module is available which offers an object-oriented approach:

>>> from pathlib import Path
>>> Path('/usr') / Path('local')
PosixPath('/usr/local')
>>> Path('/usr') / 'local'
PosixPath('/usr/local')
>>> Path('/var/log').parent
PosixPath('/var')
>>> Path('/var/log').parent.name
'var'

Callable objects

In Python, you can create a callable object not only by creating functions (with def or lambda). An object is also callable if it has the __call__ magic method:

class truncater:
    def __init__(self, length):
        self._length = length
    def __call__(self, s):
        return s[0:self._length]

print(truncater(4)('abcdabcd')) # abcd

Since a decorator is basically a higher-order function, it can also be expressed with a callable object instead of a function:

class cached:
    def __init__(self, func):
        self._func = func
        self._cache = {}
    def __call__(self, arg):
        if arg not in self._cache:
            self._cache[arg] = self._func(arg)
        return self._cache[arg]

@cached
def sqr(x):
    return x * x