Patch Your Dependencies Like a Boss
What is an automation test & why you need it?
Whenever you are writing a python application, whether it is a web app, a library, a machine-learning algorithm, etc. youâre gonna end up testing your code and Iâm not talking about manual tests buddy; weâre doing it automation test style yo!
Automation tests make future changes more manageable, in terms of what has broken so far. It also makes clear documentation of your code right out of the box. If youâre not satisfied with why the automation test is good for you, check this link and the links I am providing you at the end of this article.
Now that weâre on the same page that automation tests are awesome (we are, arenât we?), letâs delve into the more important stuff. One of which is how to patch the dependencies in a unit test.
Unit test is the first step of your automation tests and there are some arguments regarding âwhether or not we should skip them and jump to integration testâ but that is not the main topic of this article and youâre free to figure that out on your own using the links provided at the bottom of this page.
What is âpatchingâ then?đ€
To share a little bit of a personal experience, I have to say that patching dependencies have been my main concern for a long time, and now that I have finally found an elegant way to approach this problem, I felt really excited; the main reason I started writing about it here.
Now to get a feeling of what we are trying to solve here, imagine you have a service that receives input from the user, does some external API calls & after doing some processing, returns the ultimate result.
In this scenario, to test your application, that external API call has to be available every time your tests are running either locally or in your pipeline.
But what if your internet connection is too goddamn slow? Or what if the machine running your tests does not have internet access? There is also the possibility that the target website might just be down and inaccessible.
The same example applies to interacting with a database. It just might be too costly to set up a database server every time you need to test your app.
Unless youâre running integration tests, which is out of the scope of this article, nowâs a good time to address unit tests with the help of patches. These are the cases where weâll impersonate a result from a target making it look like the dependency is returning what you want it to; this makes it possible to test the actual application with that patched result.
Patching is the act of impersonating an external dependency such as a database, an API call on the internet, or even a library call, making it look like it has returned something we gave it to.
Talk is cheap, show me the code đ¶
If the above introduction doesnât make any sense, hold your horses cause weâre about to get a couple of examples below to make sure everything clicks right where it should.
As you can see from above, I needed to mock an external library (requests to be specific). This enabled me to bypass the actual request from transmitting over the internet and as a result, I get to test only the application; which is very important to me because I would like to make sure that my application is working correctly if all the external libraries are behaving seamlessly.
It also empowers thou to measure thy coverage to see if there exists a dead code or not.
The monkeypatch, the argument to the test method, is a pytest built-in fixture that you can use in every single one of your test cases should you need it. It has quite a few very useful methods that weâre going to discuss in this article. One of the most useful ones being setattr; It makes it so that any access to a class instance or class method being redirected to something completely customizable.
In the above snippet, I am fake-replacing the requests.get to return a custom object (MockStatus). This custom object has an instance attribute that I need in my code status_code. Therefore when I call the requests.get(URL).status_code itâs actually not the actual library requests.Response.status_code, but MockStatus.status_code. Beautiful, isnât it?đ»
Hereâs the list of available methods from the documentation itself:
monkeypatch.setattr(obj, name, value, raising=True)
monkeypatch.delattr(obj, name, raising=True)
monkeypatch.setitem(mapping, name, value)
monkeypatch.delitem(obj, name, raising=True)
monkeypatch.setenv(name, value, prepend=False)
monkeypatch.delenv(name, raising=True)
monkeypatch.syspath_prepend(path)
monkeypatch.chdir(path)Note: setattr instance method has 2 signatures [source]:
- The first one is as we used above using a dot-separated
modulename.classname.atttibutenameas the first argument and a custom value in the second. - The second one is passing a concrete
object(remember, everything in python is an object) as the first argument, passing a double-quoted attribute name of that object (str) as the second argument, and a custom value as the final and third argument.
Weâre gonna use both of these approaches in the next example; so stay calm, and âmay the god of understanding guide usâ. đź
Letâs get serious, do something practicalđ
Now letâs get some other examples as well while weâre at it. And this time, letâs get a little bit serious and do something more useful. shall we?
I should warn you though that the next example is a little bit longer and you might get bored in the middle. But bear with me here as there are valuable lessons that took me a long time to realize.
As you can see, this is a fully functional application with my favorite framework.
But aside from all that endless battles of âmy framework is better than yoursâ, letâs just take the lesson and move on.
I figure the source code is pretty obvious and doesnât need me doing all the talking and making this article longer than it already is, decreasing my chance of having more readers đđ.
But just to have a reference, Iâm gonna dive a little deeper into one of the test cases: test_create_new_user_successfully :
- The
clientobject is just a test application from the FastAPI itself and I didnât do any magic except making it a globally accessible fixture to all my tests in this and any future python modules. - The
monkeypatch, as mentioned earlier, is a fixture that needs no extra installation other than thepytestitself. Check the documentation for a complete API reference. So in each test case, you can see that I am mocking the database, making it âlook-likeâ the database has returned something to the application, and testing if the source code could actually work with that mocked object. - At first, I am patching the
firstmethod of SQLAlchemyQueryobject;Query.firstis equivalent to something similar to this:SELECT ⊠FROM ⊠LIMIT 1. By doing this patch weâre telling the interpreter to âassume that the database is giving you thisâ. This, in this scenario, is actuallyNone, because I donât want to get this error:Email already registered. - In the next part, I am iterating through every key-value pair of the randomly-generated-user, again telling the interpreter to fake-replace everything inside the ORM with the ones weâre giving, making it so that every access to any instance attribute would return the value we provided.
Session.commitandSession.refreshis also mocked, which might seem like an unnecessary thing to do as we have already got what we wanted inQuery.first, but doing this will remove any actual database interaction that would otherwise need an up-and-running database.- The rest of the test case is just invoking the endpoint, fetching the result, and ensuring that what we received is exactly the same as what we expected it to be; which Iâm sure youâre quite capable of figuring out but I didnât want to be presumptuous nevertheless.
Show me more, show me more, I canât waitđ
Okay now. Do you feel like itâs been enough? Or should we go for another round? I feel like you get the point, but you know what they say?
The third timeâs the charm.đ€
Nothing too fancy here. Just trying to see if setenv and delenv are doing what they are supposed to. Just remember that doing delenv in the second test case isnât actually necessary unless some evil-doer is running your test using the following command:
DATABASE_URI=i_am_not_empty pytest patch_environment.pyWhich will fail if you do not have that delenv there. We also added raising=False which avoids empty values of that key from raising KeyError.
I know itâs been enough, but this is the final round. I promise đ€.
Alright. Enough is enough. I know that youâve seen just enough to be sick of all the testing and everything. I was pretty excited about it when I first found out that there exists such a thing and this article is my appreciation to the outside world.
Time for goodbye đ„Č
Iâm a huge fan of testing so it wouldnât be inappropriate for me to do a little bit of self-promotion and tell you guys to check out my other related article as well đđ»:
And thatâs it yâall. Thanks very much for being supportive and reading till the end. Hope you enjoyed it and get something useful from it as well. And I hope you are as excited as I am (or even more) after finding out about this monkeypatch thing because I didnât know about it until now.
Please leave your thoughts or any doubts you might have in the comments below. Iâd be glad to have a chat regarding these fascinating tools. You can also mention any idea you might have regarding a new topic you want to learn, and Iâll gladly take the time to write an article for that as well. Donât worry, I wouldnât start unless I master it myself first đ .
If you want to dig even deeper, here are the links you might need.
