Mastering Python’s __getitem__ and slicing

gyu-don
gyu-don
May 17 · 6 min read

In Python’s data model, when you write foo[i] , foo.__getitem__(i) is called. You can implement __getitem__ method in your class, and define the behaviour of slicing.

Example:

class Foo:
def __getitem__(self, key):
print(key)
return None
foo = Foo()
foo[1] # => print(1)
foo["aaa"] # => print("aaa")

As you know, Python can slice more complicated object.

# comma separated slicing
d = {}
d[1, "hello"] = 3
# colon separated slicing
ls = [0, 1, 2, 3]
ls[:2]
ls[3:]
ls[2:4]
ls[::2]
ls[1:4:-1]
ls[:]
ls[::]
# both comma and colon
import numpy as np
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
a[:2, 1:3]

Let’s inspect what kind of object is passed to __getitem__ argument.

Define the class for inspection.

class Inspector:
def __getitem__(self, key):
return key

Comma Separated Slicing

First, we check comma separated slicing.

a = Inspector()
a[1] # => 1: int
a[1, 2] # => (1, 2): tuplea[1, 2, 3] # => (1, 2, 3): tuplea[] # => SyntaxError

This behaviour is very similar to right hand side of assignment expression.

t = 1 # 1: int
t = 1, 2 # (1, 2): tuple
t = 1, 2, 3 # (1, 2, 3): tuple
t = # SyntaxError

More examples needed?

a[(1, 2, 3)] # (1, 2, 3): tuple
assert a[(1, 2, 3)] == a[1, 2, 3]
t = (1, 2, 3)
u = 1, 2, 3
assert t == u
# tuple of one element.
a[1,] # (1,): tuple
t = 1, # (1,): tuple
a[(1,)] # (1,): tuple
t = (1,) # (1,): tuple
# empty tuple
a[()] # (): tuple
t = () # (): tuple
t = 1, 2
assert a[t] == a[1, 2]
a[(1, 2), 3] # ((1, 2), 3): tuple
t = (1, 2), 3 # ((1, 2), 3): tuple

However, starred item and yield expression are not allowed for slicing.

t = *iter([]), 1 # (1,): tuple
a[*iter([]), 1] # SyntaxError: invalid syntax
def g():
t = yield 1 # Valid.
a = Inspector()
a[yield 1] # SyntaxError: invalid syntax

As same as right hand side of assignment expression, comma separated slice is interpreted as tuple.

Colon separated slice

Let us inspect colon separated slice.

# As same as above part.
class Inspector:
def __getitem__(self, key):
return key
a = Inspector()

Colon separated slice makes a slice object.

a[1:2:3] # => slice(1, 2, 3)
assert a[1:2:3] == slice(1, 2, 3)

Docstring of slice object is:

slice(stop)
slice(start, stop[, step])
Create a slice object. This is used for extended slicing (e.g. a[0:10:2]).

Colon separated slice’s notifications are [start:], [start::], [:stop] , [:stop:, [::step], [start:stop], [start:stop:], [start::step] , [:stop:step] and [start:stop:step] .

start = 1
stop = 2
step = 3
assert a[start:] == a[start::] == slice(start, None, None) == slice(start, None)assert a[:stop] == a[:stop:] == slice(None, stop, None) == slice(stop)assert a[start:stop] == a[start:stop:] == slice(start, stop, None) == slice(start, stop)assert a[:stop:step] == slice(None, stop, step)assert a[start:stop:step] == slice(start, stop, step)

Now, you understood relations between colon separated slicing and slice object.

Type of start, stop, step are not only integer. Any types are acceptable.

a[[]:{}:"abc"] # => slice([], {}, "abc")class MyClass:
pass
a[MyClass():MyClass()] # => slice(MyClass(), MyClass())

slice object have start, stop, and step properties.

sl = slice(1, '2', [3])sl.start # => 1
sl.stop # => '2'
sl.step # => [3]
sl = slice('stop')sl.start # => None
sl.stop # => 'stop'
sl.step # => None

slice object have indices(len) method.

Docstring is:

S.indices(len) -> (start, stop, stride)Assuming a sequence of length len, calculate the start and stop
indices, and the stride length of the extended slice described by
S. Out of bounds indices are clipped in a manner consistent with the
handling of normal slices.

indices method returns if start , stop , step properties are not None and in the range of [0, len], returns them, otherwise, returns suitable value.

It helps to implement the slicing as same as list object.

li = [i for i in range(10)]
def make_list(start, stop, stride):
t = []
if stride > 0:
while start < stop:
t.append(start)
start += stride
else:
while start > stop:
t.append(start)
start += stride
return t
start, stop, stride = a[6:].indices(10)
# start, stop, stride = 6, 10, 1
assert li[6:] == make_list(start, stop, stride) == [6, 7, 8, 9]
start, stop, stride = a[:7:2].indices(10)
# start, stop, stride = 0, 7, 2
assert li[:7:2] == make_list(start, stop, stride) == [0, 2, 4, 6]
start, stop, stride = a[:].indices(10)
# start, stop, stride = 0, 10, 1
assert li[:] == make_list(start, stop, stride) == [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
start, stop, stride = a[15:].indices(10)
# start, stop, stride = 10, 10, 1
assert li[15:] == make_list(start, stop, stride) == []
start, stop, stride = a[-3:].indices(10)
# start, stop, stride = 7, 10, 1
assert li[-3:] == make_list(start, stop, stride) == [7, 8, 9]
start, stop, stride = a[:5:-2].indices(10)
# start, stop, stride = 9, 5, 2
assert li[:5:-2] == make_list(start, stop, stride) == [9, 7]

Colon separated slicing returns slice object. It has 3 properties. start, stop and step. If you want to implement the slicing as same as list object’s implementation, slice.indices method is helpful.

Both comma and colon separated slicing

In this case, it is interpreted as tuple of slice.

class Inspector:
def __getitem__(self, key):
return key
a = Inspector()a[1:2,3:4] # (slice(1, 2, None), slice(3, 4, None))
a[1,2:3,4] # (1, slice(2, 3, None), 4)
a[1:(2,3):4] # slice(1, (2, 3), 4)

__index__ method

__index__ method is very important to be a slicing master.

import numpy as npi = np.int32(3)
print(i) # => '3'
print(type(i)) # => 'np.int32'

In this code, is ia integer?

Yes. It is an integer type object but not a Python’s builtin int type object.

It may cause some troubles in your __getitem__ implementation. You may want to convert to Python’s builtin int type.

You may think int(i) returns builtin int type. Yes, it is commonly used in Python. However, if i is floating point number, you may not want to convert to integer value. In this case, __index__ method is useful.

object.__index__(self)
Called to implement operator.index(), and whenever Python needs to losslessly convert the numeric object to an integer object (such as in slicing, or in the built-in bin(), hex() and oct() functions). Presence of this method indicates that the numeric object is an integer type. Must return an integer.

Note
In order to have a coherent integer type class, when __index__() is defined __int__() should also be defined, and both should return the same value.
(Python documentation Data model)

numpy.int{8, 16, 32, 64}, uint{8, 16, 32, 64} implements __index__ method and it returns Python’s builtin int type. float , numpy’s floating point numbers are not implements __index__ method.

When you want to convert an variable to int type for slicing, you should use this method.

You can also use operator.index function. When __index__ is not implemented foo.__index__() raises AttributeError, operator.index(foo) raises TypeError .

import operator
import numpy as np
i = np.int32(3)
i.__index__() # Returns 3: int
operator.index(i) # Returns 3: int
# builtins int type also implements __index__ method.
i = 3
i.__index__() # Returns 3: int
operator.index(i) # Returns 3: int
# floating point type doesn't implement __index__ method.
i = 3.0
i.__index__() # AttributeError: 'float' object has no attribute '__index__'
operator.index(i) # TypeError: 'float' object cannot be interpreted as an integer

__setitem__ and __delitem__

__setitem__ and __delitem__ are similar method with __getitem__ .

foo.__setitem__(key, value) is called when foo[key] = value is evaluated. foo.__delitem__(key) is called when del foo[key] is evaluated. You may implement these methods for your class.

Conclusion

In this article, we learned following.

  • foo[key] calls foo.__getitem__(key)
  • foo[key] = value calls foo.__setitem__(key, value)
  • del foo[key] calls foo.__delitem__(key)
  • foo[k1, k2] calls foo.__getitem__((k1, k2))
  • foo[k1:k2:k3] calls foo.__getitem__(slice(k1, k2, k3))
  • Integer type implements __index__ method

This knowledge is useful for implementation of your subscriptable class.

gyu-don

Written by

gyu-don

Takumi Kato, quantum computer engineer.