Mocking during Python Unit Testing with Azure Event Hubs and SQL Database

Why Unit Test?

While creating the structure of this article I was thinking why to have this section, writing Unit Test for the code is so obvious now a days and kind of enforced via Pull Request validation by measuring Code Coverage of Unit Test.

But I have seen several time we often miss the crucial aspect on why to write Unit Test. In my opinion the primary objective of Unit Test should be testing of the code/logic for myself and tell me — if I have written the function/class as expected, if I provide input am I getting desired output from a function/class/code blocks, is it throwing exception as designed, etc. That will help me to get the earliest feedback even before actual integration/execution.

Why Mocking?

It is not very obvious that all the resources/dependencies (Azure SQL Database or Azure Event Hubs, etc.) will be available while executing Unit Tests (even if available, it will not be cost effective to create resources/dependencies only for Unit Test as Unit Tests will be executed by different developers/CI very frequently).

Mocking can help us to overcome this problem, the resources/dependencies can be replaced with mock objects during Unit Tests execution.

But the question is if we mock all the resources/dependencies how the logic written on top this will be tested? Patching is the way. With patch decorator it is possible to produce predefined results (that is supposed to be produced during real execution by the resources/dependencies) that will enable assertions of the logic written on top of it during Unit Test — aha not very clear, I know, let’s see the below use cases, hope that will clarify.

Unit Testing with Azure Event Hubs

Let’s take simple logic where we need to write a receiver of Azure Event Hubs Python SDK -

Following program represents a class MyEventHub, has a function receive_events() that receives events from Azure Event Hubs for a certain interval and accept/drop events based on some conditions and finally returns list of accepted events along with properties.

Now we need to write Unit Tests of it. We will see how we can use patch decorator to replace functions like from_connection_string() with Mock object and perform Unit Test without actual Azure Event Hubs resource, where without calling Azure Event Hubs every other logic will be Unit Tested (related discussion) -

Unit Testing with Azure SQL Database

We will see some more advance usage of Mock and patch, while writing Unit Tests for Azure SQL Database. Let’s take a look to the following example -

In this program there are two classes -

MyDBOperation: responsible for connecting to Azure SQL Database and has a function get_some_valid_items() that queries into Database, do some filtering and finally returns a list of dictionaries of key-value pairs, where key is the Database column name and values are from different rows

MyAnalysis: contains function analysis() that uses class MyDBOperation and perform some analysis on the items returned by get_some_valid_items()

And we need to write Unit Tests for class MyAnalysis. We can patch pyodbc.connect and we will be able to write Unit Tests. But that will not cover analysis() function in class MyAnalysis. And if we patch pyodbc.connect then function get_some_valid_items() of class MyDBOperation will return a non-iterable Mock object instead of a list. Hence we will not be able to test the logic inside for loop in function analysis() in class MyAnalysis.

To solve this problem we need to use return_value() of Mocked object and then we can assign the required fields/functions/objects with expected values and then rest of logic can be tested -

In this example we have modified the cursor object of the Mocked response and injected description field with the column names of the Database and fetchall() function with one dummy row. Now while function analysis() is getting executed some_items will be a list containing one item and we are able to test the logic inside the for loop.

Conclusion:

Let’s look at the following diagram that illustrates what we have seen so far -

As we can see if we use Mock we are able to remove the dependencies and able to test without actual resources/dependencies. Also by injecting expected output from the dependencies using return_value() of the Mocked object we are able to test the logic written on top of the outputs from the dependencies.

I hope I was able to clarify some aspects of Unit Testing with Azure Event Hubs and SQL Database, happy Unit Testing😊!

“Quality is never an accident; it is always the result of intelligent effort.” – John Ruskin