Right Way to Test, Mock, and Patch in Python

Munish Goyal
Feb 1 · 21 min read

Instead of TDD (Test Driven Development) which forces you to think about tests first, one can practice TPD (Test Paralleled Development). In TPD, you start with code development, but during development, you check the correctness of the code by writing tests and executing them (instead of running the code directly or using the console).

Photo by David Clode on Unsplash

Table of Contents

· The unittest module: Unit Testing Framework
∘ _The unittest Basics
∘ _Running Tests using unittest Module
· The unittest.mock module: Mock Object Library
∘ _Mock Class: Mocking objects and/or an attribute
∘ _The MagicMock Class
∘ _Patching imports with patch
∘ _Mock Helpers

The unittest module: Unit Testing Framework

References:

The unittest Basics

Quick Notes:

  • A testcase is a class representing logical test unit for group of related tests (such as for a class, or a module, or even a function). It is instance of subclass unittest.TestCase.
  • A test is a method (within a testcase) whose name start with letters test (but generally test methods are prefixed with test_). This naming convention indicates to the test runner about which methods represents tests.
  • The crux of each test is a call to assert<Something>() methods such as following (and many more). So that the test runner can accumulate all test results, we use these methods instead of assert statement:
  • self.assertEqual(a, b),
  • self.assertNotEqual(a, b),
  • self.assertTrue(x),
  • self.assertFalse(x),
  • self.assertIsNone(x),
  • self.assertIsNotNone(x),
  • self.assertIs(a, b),
  • self.assertIsNot(a, b),
  • self.assertIn(a, b),
  • self.assertNotIn(a, b),
  • self.assertIsInstance(a, b),
  • self.assertNotIsInstance(a, b),
  • self.assertRaises(),
  • self.assertRaisesRegex(),
  • self.assertWarns(),
  • self.assertWarnsRegex(),
  • self.assertLogs(),
  • The setUpClass() and tearDownClass() allows you to define instructions that will be run before and after an individual testcase is run.
  • Use the setUp() and tearDown() methods we can define instructions that will be executed before and after each test. Particular, you can set instance variables in setUp(), which can then be used in test_s.
  • The decorators such as @unittest.skip("reason") and @unittest.skipIf(condition) can be used to skip tests.

For example,

Running Tests using unittest Module

The unittest module can be used from the CLI to run:

  • tests from modules,
  • tests from classes, or
  • even individual test methods

You pass it with modules names, or fully qualified class names, or fully qualified method names, or via file_path of the test module as well.

For example,

Specifying modules as files allows us to use the shell filename completion to specify the test modules. The specified file must be still importable as a module. The path is converted to a module name by removing the .py and converting path separators into . (dot). If we want to run a test file that isn’t importable as a module we should run the file directly instead.

Some of the important flags while running tests are:

  • -f (or --failfast): stop the test run on the first error or failure
  • -v (or --verbose): for verbosity
  • -b (or --buffer): for suppressing the output of print statements
  • -s (for nosetests): nosetests don’t show output by default; use -s option to not to supress output
  • -x (or --stop) (for nosetest): for enabling failfast on nosetest

For a list of all the command-line options, use -h (or --help) option:

python -m unittest -h

The unittest.mock module: Mock Object Library

References:

The mock is a Python library to create mock objects, which helps us in replacing parts of your system under test with mock objects and set assertions about how they have been used. It is part of Python standard library, available as unittest.mock in Python 3.3 onwards. [1]

The Mock class of unittest.mock removes the need to create a host of stubs throughout your test suite. After performing certain action, we can set assertions about which methods/attributes were used and arguments they were called with. It lets us specify return values and set needed attributes. The magic methods can be handled through MagicMock, which is subclass of Mock. The Mock and MagicMock objects on-the-fly create attributes and methods as we access them and record details of how they have been used. [1]

Mocks is based on the action -> assertion (that is, first let the mock be used and then make assertions on it about the calls it received) pattern instead of record -> replay pattern used by many mocking frameworks. [1]

On top of this, mock module provides a decorator patch() which handles patching module and class level attributes within the scope of a test, along with sentinel for creating unique objects.

from unittest import TestCase, mock
# mock.Mock, mock.MagicMock, mock.patch, mock.call, mock.sentinel, mock.DEFAULT

Mock Class: Mocking objects and/or an attribute

A Mock‘s object is used to mock an object, including functions, (with all of its data and functional attributes), a particular attribute (data or functional) of an existing object, and allows to perform actions on that mock which are recorded and can be inspected later.

Note: A class can also be mocked, but in that case you would like to specify its mock’s return_value to return another mock representing its object.

The main characteristic and the default behavior of a Mock object is that it will return another Mock instance when:

  • calling the object itself
  • accessing one of its attributes (or call method)

This is the default behavior, but it can be overridden in different ways. For example, you can assign a value to an attribute of the Mock by:

  • assigning it directly, like you’d do with any Python object, or
  • pass keyword arguments to the Mock class at the time of creation, or
  • use the configure_mock method on an instance

For example,

mock = mock.Mock()mock.foo = 'bar'
assert mock.foo == 'bar'
mock.configure_mock(bar='baz')
assert mock.bar == 'baz'
mock = mock.Mock(foo='bar')
assert mock.foo == 'bar'

To override calls to the mock we would need to configure its return_value property (which is also available as a keyword argument in the Mock initializer). The Mock created this way always return the same value on all calls, but again this can be configured by using the side_effect attribute:

  • if you’d like to return different values on each call, you can assign an iterable to side_effect
  • if you’d like to raise an exception when calling the Mock, you can simply assign the exception object to side_effect

For example,

mock = mock.Mock()
mock.return_value = 42
assert mock() == 42
mock.side_effect = ['foo', 'bar', 'baz']
assert mock() == 'foo'
assert mock() == 'bar'
assert mock() == 'baz'
try:
mock()
except StopIteration:
assert True
else:
assert False
mock.side_effect = RuntimeError('Boom')
try:
mock()
except RuntimeError:
assert True
else:
assert False

The Mock objects record all interactions with them and you can then inspect these interactions. For example, you can use call_args_list attribute to inspect the arguments to all of the calls to the object as a callable, and method_calls attribute to inspect all of its method calls. You can use the reset_mock() method to clear the saved interactions (just the interactions will be rest, but not the configurations).

For example,

mock = mock.Mock()mock(1, foo='bar')      #=> <Mock name='mock()' id='4522261528'>
mock.foo() #=> <Mock name='mock.foo()' id='4524858840'>
mock(2, bar='foo') #=> <Mock name='mock()' id='4522261528'>
mock.foo(bar=123) #=> <Mock name='mock.foo()' id='4524858840'>
mock.call_args_list #=> [call(1, foo='bar'), call(2, bar='foo')]
mock.method_calls #=> [call.foo(), call.foo(bar=123)]
mock.reset_mock()
mock.call_args_list #=> []
mock.method_calls #=> []

An example on attaching mock attribute to an existing object:

We can set the attribute directly in the instance. This is fairly safe as the change is limited to the single instance.

class ProductionClass: pass
mock = mock.Mock(name='foo', return_value='bar')
thing = ProductionClass()
thing.method = mock
thing.method(1, 2, k='v') #=> 'bar'mock.call_args_list #=> [call(1, 2, k='v')]
mock.method_calls #=> []

Alternatively, you can use patch.object.

The general form of the call to Mock class is:

from unittest import TestCase, mock
# mock.Mock, mock.MagicMock, mock.patch, mock.call, mock.sentinel, mock.DEFAULT
mock.mock.Mock(
spec=None,
side_effect=None,
return_value=DEFAULT,
wraps=None,
name=None,
spec_set=None,
unsafe=False,
**kwargs
)

Here,

  • the name: If the mock is provided with a name then it will be used as the representation string (repr) of the mock. This can be useful for debugging. The name is propagated to child mocks.

For example,

mock = mock.Mock(name='foo')
mock #=> <Mock name='foo' id='4490412160'>
mock() #=> <Mock name='foo()' id='4490412160'>
dir(mock)
# ['assert_any_call',
# 'assert_called',
# 'assert_called_once',
# 'assert_called_once_with',
# 'assert_called_with',
# 'assert_has_calls',
# 'assert_not_called',
# 'attach_mock',
# 'call_args',
# 'call_args_list',
# 'call_count',
# 'called',
# 'configure_mock',
# 'method_calls',
# 'mock_add_spec',
# 'mock_calls',
# 'reset_mock',
# 'return_value',
# 'side_effect']
  • the return_value: This is the value returned when the mock is called. If it is not specified, then by default this is a new Mock (created on first access). See the return_value attribute.

For example,

mock = mock.Mock(return_value=3)
mock() #=> 3
mock = mock.Mock()
mock() #=> <Mock name='mock()' id='4490413952'>
mock.return_value() #=> <Mock name='mock()()' id='4490412272'>
mock.return_value.assert_called_with()
mock.return_value.foo = 'bar'
mock.return_value() #=> <Mock name='mock()()' id='4490412272'>
mock.return_value.foo #=> 'bar'
  • the spec: This argument be either a list of strings or an existing object (a class as an object or an instance) which acts as the specification for the mock object. Accessing any attribute not in the list or not an attribute of the passed object (that is, not present in list formed by calling dir() on the passed object), will raise an AttributeError. But note that if spec is not specified, then AttributeError is raised only for unavailable special methods.

If spec is an object (not a list of strings) then a call to __class__ would return the class of the spec object. This allows mocks to pass isinstance() tests. Generally, the __class__ attribute of an object will return its type but for a mock object with a spec, the call to __class__ would return the spec class (or spec’s class) instead. This permits the mock objects to pass the isinstance() tests for the object they are replacing/masquerading. [1]

mock = mock.Mock()
mock.foo() #=> <Mock name='mock.foo()' id='4525070992'>
mock.bar(foo=1) #=> <Mock name='mock.bar()' id='4522741040'>
mock.__str__ #=> <method-wrapper '__str__' of Mock object at 0x10db92e10>
mock.__str2__ # AttributeError: __str2__
mock = mock.Mock(spec=3)
type(mock) #=> <class 'unittest.mock.Mock'>
mock.__class__ #=> <class 'int'>
isinstance(mock, int) #=> True
isinstance(mock, Mock) #=> True
mock = mock.Mock(spec=int)
mock.__class__ #=> <class 'int'>
dir(mock) #=> ['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'assert_any_call', 'assert_called', 'assert_called_once', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'assert_not_called', 'attach_mock', 'bit_length', 'call_args', 'call_args_list', 'call_count', 'called', 'configure_mock', 'conjugate', 'denominator', 'from_bytes', 'imag', 'method_calls', 'mock_add_spec', 'mock_calls', 'numerator', 'real', 'reset_mock', 'return_value', 'side_effect', 'to_bytes']dir(int) #=> ['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']mock.foo() # AttributeError: Mock object has no attribute 'foo'mock.__str__ #=> <method-wrapper '__str__' of Mock object at 0x12ad6c978>mock.__str2__ # AttributeError: Mock object has no attribute '__str2__'

The __class__ is assignable to, this allows a mock to pass an isinstance() check without forcing you to use a spec attribute, which you would generally prefer to do if you are using list form for spec. For example,

mock = mock.Mock(name='without_spec', spec=['aa', 'ab', 'ac'])
mock.__class__ #=> <class 'unittest.mock.Mock'>
mock.__class__ = int
isinstance(mock, int) #=> True
mock.__class__ #=> <class 'int'>
dir(mock) #=> ['aa', 'ab', 'ac', 'assert_any_call', 'assert_called', 'assert_called_once', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'assert_not_called', 'attach_mock', 'call_args', 'call_args_list', 'call_count', 'called', 'configure_mock', 'method_calls', 'mock_add_spec', 'mock_calls', 'reset_mock', 'return_value', 'side_effect']

side_effect: It is either a function which is to be called when the mock is called, an iterable or an exception (class or instance) that is to be raised. See the side_effect attribute.

  • In case a function is passed, the passed function will be called with same arguments as the mock and unless the function returns the DEFAULT singleton , in which case a call to the mock will then return whatever the function returns. [1]
  • If the function returns DEFAULT then the mock will return its normal value (from the return_value).
  • If an iterable is passed, it is used to get an iterator which must yield a value on every call. This value can either be an exception instance to be raised, or a value to be returned from the call to the mock (DEFAULT) handling is identical to the function case). [1]

For example,

fn_side_effect = lambda value: value + 1
mock = mock.Mock(side_effect=fn_side_effect)
mock(3) #=> 4
mock = mock.Mock(return_value=3)
def fn_side_effect(*args, **kwargs): return mock.DEFAULT
mock.side_effect = fn_side_effect
mock() #=> 3
mock = mock.Mock()
mock.side_effect = [3, 2, 1]
mock() #=> 3
mock() #=> 2
mock() #=> 1
mock() # StopIteration
mock = mock.Mock(side_effect=Exception('Boom!'), return_value=3)
mock() # Exception: Boom!
mock() # Exception: Boom!
mock.side_effect = None
mock() #=> 3

Setting the side_effect to None clears it.

THE CONFIGURE_MOCK() METHOD TO ADD ARBITRARY ATTRIBUTES ON EXISTING MOCK OBJECT

You can also call a Mock with arbitrary keyword arguments. This way you can set attributes of the mock post it is created.

The configure_mock()‘s general form is:

configure_mock(**kwargs)

It sets attributes on the mock through keyword arguments. Attributes return values and their side effects can be set on child mocks using dot notation and by unpacking a dictionary in the method call. [1]

For example,

mock = mock.Mock()
attrs = {'method.return_value': 3, 'other.side_effect': Exception('Boom!')}
mock.configure_mock(**attrs)mock.method() #=> 3
mock.other() # Exception: Boom!

We can get the same thing by using constructor call to mocks. For example,

attrs = {'method.return_value': 3, 'other.side_effect': Exception('Boom!')}mock = mock.Mock(foo='bar', **attrs)mock.foo                #=> 'bar'
mock.method() #=> 3
mock.other() # Exception: Boom!

The configure_mock() make it convenient to do configuration after the mock has been created.

MOCK ASSERTION METHODS

<mock>.assert_called():

Use it to assert that the mock was called at least once.

<mock>.assert_called_once():

Use it to assert that the mock was called exactly once.

<mock>.assert_any_call(*args, **kwargs):

Use it to assert that the mock has been called with specified arguments. The assertion passes if the mock is atleast called once, unlike assert_called_with() and assert_called_once_with() that only pass if the call is the most recent one, and in the case of assert_called_once_with() it must also be the only call recorded.

For example,

mock = mock.Mock(return_value=3)
mock(1, 2, 3) #=> 3
mock(4, 5, 6) #=> 3
mock.assert_called() #=> None
mock.assert_any_call(1, 2, 3) #=> None
mock.assert_any_call(4, 5, 6) #=> None
mock.assert_any_call(7, 8, 9) # AssertionError: mock(7, 8, 9) call not found
mock.assert_any_call() # AssertionError: mock() call not found
mock.assert_called_with(4, 5, 6) #=> None
mock.assert_called_with(1, 2, 3) # AssertionError: Expected call: mock(1, 2, 3), Actual call: mock(4, 5, 6)
mock.assert_called_once_with(4, 5, 6) # AssertionError: Expected 'mock' to be called once. Called 2 times.
mock = mock.Mock(return_value=3)
mock(4, 5, 6) #=> 3
mock.assert_called_once_with(4, 5, 6) #=> None
mock.assert_not_called() # AssertionError: Expected 'mock' to not have been called. Called 1 times.
mock.reset_mock() #=> None
mock.assert_not_called() #=> None
mock = mock.Mock(return_value=3)
mock.assert_not_called() #=> None
mock([{'foo': 'bar'}, 2]) #=> <Mock name='mock()' id='4489369416'>
mock.assert_any_call([{'foo': 'bar'}, 2]) #=> None
mock.assert_called_with([{'foo': 'bar'}, 2]) #=> None

<mock>.assert_called_with(*args, **kwargs):

This method is used to assert that the recent call is made with specified arguments. With the given arguments, this call should be the most recent call on the given mock object.

<mock>.assert_called_once_with(*args, **kwargs):

It asserts that the mock was called exactly once and that too with the specified arguments, this call should be the most recent call on the given mock object.

<mock>.assert_has_calls(calls, any_order=False):

It asserts the mock has been called with the specified calls. The mocked_calls list is checked for the calls. If any_order is:

  • falsy (the default): then the calls must be in order. There can be extra calls made before or after the specified calls
  • truthy: then the calls can be in any order, but all those calls must all appear in mock_calls.
mock = mock.Mock(return_value=None)
mock(1)
mock(2, 5)
mock(7)
mock.assert_has_calls([call(1), call(2, 5)]) #=> Nonemock.assert_has_calls([call(2, 5), call(1)]) # AssertionError: Calls not found. Expected: [call(2, 5), call(1)], [call(1), call(2, 5), call(7)]mock.assert_has_calls([call(2, 5), call(1)], any_order=True) #=> None

THE <MOCK>.RESET_MOCK() METHOD

It is used to reset all the call attributes on a mock object. That is, call_args_list and method_calls will become empty list. You can use it to make a series of assertions that reuse the same object.

Note that reset_mock() doesn’t clear the configurations (such as return_value, side_effect or any child attributes you have using normal assignment by default), but only clears the interactions.

THE MOCK_CALLS ATTRIBUTE

The mock_calls records all calls to the mock object, its methods, magic methods, and return value mocks. [1]

To understand the difference between mock_calls, call_args_list, and method_calls, check following example:

mock = mock.MagicMock()
mock(1, 2) #=> <MagicMock name='mock()' id='4524600904'>
mock.meth1(foo='bar') #=> <MagicMock name='mock.meth1()' id='4522738296'>len(mock) #=> 0
mock.mock_calls #=> [call(1, 2), call.meth1(foo='bar'), call.__len__()]
mock.call_args_list #=> [call(1, 2)]
mock.method_calls #=> [call.meth1(foo='bar')]

THE CALL_ARGS_LIST ATTRIBUTE

The call_args_list returns list of all calls made to the mock object in sequence (so the length of the list is the number of times it has been called, and the first call at index 0). Before any calls have been made it returns empty list. The call_args() (that is call_args_list[0]) is either None (if the mock hasn’t been called), or the argument that the mock was last called with. [1]

Members of call_args_list are call() object. We can unpacked these tuples to get at the individual arguments. The call() can be used for conveniently constructing lists of calls to compare with call_args_list.

For example,

THE METHOD_CALLS ATTRIBUTE

As well as tracking to themselves, mocks also track calls to methods and methods of its attributes. Use method_calls for this purpose.

For example,

mock = mock.Mock(return_value=None)mock.method_1()                             #=> <Mock name='mock.method()' id='4489637224'>mock.method_2()                             #=> <Mock name='mock.method_2()' id='4489510640'>mock.some_attribute                         #=> <Mock name='mock.some_attribute' id='4489507952'>mock.another_attribute                      #=> <Mock name='mock.another_attribute' id='4489370816'>mock.yet_another_attribute.some_method()    #=> <Mock name='mock.yet_another_attribute.some_method()' id='4489751968'>mock.method_calls                           #=> [call.method_1(), call.method_2(), call.yet_another_attribute.some_method()]mock.call_args_list                         #=> []

MOCKING MAGIC METHODS

Python’s protocol methods (also called magic methods) can also be mocked with Mock.

We mock a magic method by setting the method to be mocked to a function or a mock instance. In case of a function, it must take self as the first argument because the function is basically hooked up to the class, but each Mock instance is kept isolated from the others.

class MyClass: passMyClass.__str__ = __str__
obj = MyClass()
str(obj) #=> 'fooble'
mock = mock.Mock()
mock.__str__ = __str__
str(mock) #=> 'fooble'
mock = mock.Mock()
mock.__str__ = Mock()
mock.__str__ = Mock(return_value='fooble')
str(mock) #=> 'fooble'

One of the use case for this is for mocking the objects used as context managers in a with statement. For example,

mock = mock.Mock()
mock.__enter__ = Mock(return_value='foo')
mock.__exit__ = Mock(return_value='bar')
with mock as m:
assert m == 'foo'
mock.__enter__.assert_called_with()
mock.__exit__.assert_called_with(None, None, None)

Note: If we use the spec keyword argument to instantiate a mock then setting a magic method that isn’t in the spec will raise an AttributeError error.

Note: Calls to magic methods do not appear in method_calls, but they are recorded in mock_calls.

A full list of supported magic methods is:

  • __hash__, __sizeof__, __repr__ and __str__
  • __dir__, __format__ and __subclasses__
  • __floor__, __trunc__ and __ceil__
  • Comparisons: __lt__, __gt__, __le__, __ge__, __eq__ and __ne__
  • Container methods: __getitem__, __setitem__, __delitem__, __contains__, __len__, __iter__, __reversed__ and __missing__
  • Context manager: __enter__ and __exit__
  • Unary numeric methods: __neg__, __pos__ and __invert__
  • The numeric methods (including right hand and in-place variants): __add__, __sub__, __mul__, __matmul__, __div__, __truediv__, __floordiv__, __mod__, __divmod__, __lshift__, __rshift__, __and__, __xor__, __or__, and __pow__
  • Numeric conversion methods: __complex__, __int__, __float__ and __index__
  • Descriptor methods: __get__, __set__ and __delete__
  • Pickling: __reduce__, __reduce_ex__, __getinitargs__, __getnewargs__, __getstate__ and __setstate__

The MagicMock Class

MagicMock is a direct subclass of Mock that implements default magic methods. This makes MagicMock useful in mocking class behavior, which is why it’s the default class when patching.

To understand the difference between Mock and MagicMock, check following example:

mock = mock.Mock()
len(mock) # TypeError: object of type 'Mock' has no len()
mock.mock_calls #=> []
mock = mock.MagicMock()
len(mock) #=> 0
mock.mock_calls #=> [call.__len__()]

As we can see, mock_calls attribute was used to list calls to magic methods.

MagicMock is direct subclass of Mock with default implementations for most of the dundar (magic) methods. You can use mock.MagicMock without having to configure the magic methods yourself.

The constructor parameters have the same meaning for the attributes as that for Mock.

The magic methods are setup with mock.MagicMock object, so you can configure them and use them in the usual way:

mock = mock.MagicMock()mock[3] = 'fish'mock.__setitem__.assert_called_with(3, 'fish')mock.__getitem__.return_value = 'result'mock[2]                           #=> 'result'

Patching imports with patch

WHY, WHAT, AND HOW TO PATCH?

Why to Patch?

The primary reason to use unittest.mock.patch is to patch imports in the module under test using the patch function. The patch intercepts import statement, and return a Mock instance that we can preconfigure using the methods we discussed earlier.

For example, imagine we want to test this very simple function:

import osdef work_on():
path = os.getcwd()
print(f'Working on {path}')
return path

Note that we’re importing os and calling getcwd to get the current working directory. We don’t want to actually call it on our tests though because it might be expensive to make this call on underneath system (such as calling an external API endpoint) or if we don’t want to mark our method to be a failure if call to os.getwd() breaks for some reason.

What to Patch?

We can supply patch with a string representating specific import. We do not want to simply supply os.getwd because before the test is run, the import statement for work in the test file would already bring in os module into work module, so patching os within test might not have any affect on os.getcwd reference used by work module. So, instead we want to supply the module under test’s import of os, i.e, work.os. When this module is imported patch will return a Mock instead.

Note that, target can be any item which can be imported form a module (or a module from package). So, generally it is either a class or a function (not any class attribute or method). If an object of the class is created in the code under test then it will be the return_value of the mock that will be used.

The patch() works by temporarily changing the object that a name points to with another one. There may exist many names pointing to same individual object, but for patching to work we make sure that we patch the name used by the system under test, and before the original definition is already assigned to attribute or variable under the test.

We must note that, we patch only where an object is looked up, which is not necessarily the same place as where it is defined.

For example, consider following project structure that we want to test:

a.py
-> defines `AClass`
b.py
-> `from a import AClass`
-> `a_func` instantiates `AClass`
-> we want to test `a_func`

Now, we want to test a_func but we want to mock out AClass using patch(). The problem is that when b.py is loaded then the execution of from a import AClass imports AClass from module a. So, if we use patch() to mock out a.AClass in our test, then it will have no effect on our test; module b already has a reference to real AClass and so patching will not have any effect. So, here we need to patch out AClass where it is used (or where it is looked up). In this case a_func will actually look up AClass in module b, where we have imported it. The patching should look like: @patch('b.AClass').

a.py
-> defines `AClass`
b.py
-> `import a`
-> `a_func` uses `a.AClass`
-> we want to test `a_func`

If instead of from a import AClass, the module b does import a and a_func uses a.AClass. In this case the class we want to patch is being looked up in the module and so we have to patch a.AClass instead: @patch('a.AClass').

How to Patch?

The straight-forward way to patch is to use patch() method as context manager, with a with statement.

The patch() can be used as a context manager, with the with statement. In this case, patching is applied to the indented block after the with statement. If we use as then the patched object will be bound to the name after the as; which is useful if patch() is creating a mock object for us.

For example,

This can be testing easily by running as python -m unittest just_random:

As an alternative and convenient way, we can use the decorator version of patch. But, this time the test will have an extra parameter: mocked_os to which the Mock is injected into the test.

The patch will forward keyword arguments to the Mock class, so to configure a return_value we simple add it as one:

Otherwise, one can use start() and stop() methods as well. But this is not generally recommended.

The patch() has start() and stop() methods. Using these methods we can easily do the patching in setUp methods or where we want to do multiple patches without nesting the decorators or with statements.

To use them, we call patch() as normal and keep a reference to the returned patcher object. We can then call start() to put the patch in place and stop() to return to original behavior.

For example,

patcher = mock.patch('package.module.ClassName')
from package import module
original = module.ClassName
new_mock = patcher.start()
assert module.ClassName is not original
assert module.ClassName is new_mock
patcher.stop()assert module.ClassName is original
assert module.ClassName is not new_mock

If you use this technique you must ensure that the patching is removed by calling stop(). This can be complicated than you might think, because if an exception is raised in the setUp then tearDown is not called. The unittest.TestCase.addCleanup() makes this easier.

It is also possible to stop all patches which have been started (using start()) by using patch.stopall().

The patch‘s general form is:

from unittest import TestCase, mock
# mock.Mock, mock.MagicMock, mock.patch, mock.call, mock.sentinel, mock.DEFAULT
mock.patch(
target,
new=DEFAULT,
spec=None,
create=False,
spec_set=None,
autospec=None,
new_callable=None,
**kwargs
)

Here, [1]

  • The patch() acts as a function decorator, class decorator, or a context manager. Within the body of the function or while using with statement, the target is patched with a new object. When the function or with statement exits the patch is removed.
  • The target should be a string of the form 'package.module.ClassName'. The target is imported and the object it specifies is replaced with the new object, so the target must be importable from the environment you are calling patch() from. We must note that the target is imported when the decorated function is executed, not at the decoration time.
  • If new is not specified, then the target is replaced with MagicMock. If patch() is used as a decorator and new is not passed, then the created mock is passed as an extra argument to the decorated function. If patch() is used as a context manager, the created mock is returned by the context manager. [1]
  • The spec and spec_set keyword arguments are passed to the MagicMock if patch is creating one for us. We can also set spec or spec_set to True, which would cause the pass in the object being mocked as the spec/spec_set object. A more powerful form of spec is autospec. See the create_autospec() and Autospeccing. [1]
  • The new_callable lets you specify a different class, or callable object, that will be called to create the new object. By default MagicMock is used. [1]

By default, patch() would fail to replace attributes that don’t exist already. If we set create as True, and if the attribute doesn’t exist, patch will create the attribute for us when the patched function is called, and delete it again afterwards. [1]

Patch can be used as a SomeTestClass decorator. It decorates each test method in the class. This reduces the boilerplate code when we test methods which share a common patching set.

Patching Builtins:

The builtins functions (such as print) can be patched as mock.patch('builtins.<function_name>'). For example following snippet:

THE PATCH.OBJECT()

The patch.object is similar to patch, but is used to patch a particular attributes (and keeping other attributes intact). For example,

@mock.patch.object(SomeClass, 'class_method')

Mock Helpers

MOCK HELPER SENTINEL

Sometimes during testing we need to test that a specific object (that is, with the same id) is passed as an argument to another method, or returned. In such cases, we generally resort to using some named objects. The sentinel provides an easier way of creating and testing the identity of objects like this.

It creates the attributes on demand when we access them by name. Accessing an attribute multiple times would always return the same object. The objects returned have a sensible representation (defined by repr) so that the test failure messages are readable.

Starting from Python 3.7, the sentinel attributes preserve their identity when they are copied or picked.

For example,

class ProductionClass: passreal = ProductionClass()
real.method = mock.Mock(name="method")
real.method.return_value = mock.sentinel.some_object
result = real.method()
assert result is mock.sentinel.some_objectmock.sentinel.some_object #=> sentinel.some_object

MOCK HELPER DEFAULT

The DEFAULT object is a pre-defined sentinel (actually sentinel.DEFAULT). It is generally used by side_effect functions to hint that the normal return value should be used.

MOCK HELPER CALL

The call(*args, **kwargs) is a helper for making simpler assertions, for comparing with call_args_list, method_calls, mock_calls, and call_args. The call() can also be used with assert_has_calls().

An example of how call objects can be inspected:

mock = mock.Mock()mock(1, 2, 3)                     #=> <Mock name='mock()' id='4891259904'>mock(3, k=1)                      #=> <Mock name='mock()' id='4891259904'>mock.meth1(4)                     #=> <Mock name='mock.meth1()' id='4891260352'>mock.meth1(name='foo')            #=> <Mock name='mock.meth1()' id='4891260352'>mock.method_calls                 #=> [call.meth1(4), call.meth1(name='foo')]mock.method_calls[0]              #=> call.meth1(4)mock.method_calls[1]              #=> call.meth1(name='foo')mock.method_calls[1][0]           #=> 'meth1'mock.method_calls[1][1]           #=> ()mock.method_calls[1][2]           #=> {'name': 'foo'}mock.method_calls[1][3]           # IndexError: tuple index out of range'meth1' in mock.method_calls[1]   #=> True

References:

[1]: Python’s unittest.mock module, https://docs.python.org/3/library/unittest.mock.html

Geek Culture

Proud to geek out.

Thanks to The Startup

Sign up for Geek Culture Hits

By Geek Culture

Subscribe to receive top 10 most read stories of Geek Culture — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Munish Goyal

Written by

Designing and building large scale applications/APIs, ambitious data models, and workflows! https://goyalmunish.github.io

Geek Culture

A new tech publication by Start it up (https://medium.com/swlh).

Munish Goyal

Written by

Designing and building large scale applications/APIs, ambitious data models, and workflows! https://goyalmunish.github.io

Geek Culture

A new tech publication by Start it up (https://medium.com/swlh).

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store