Testing (almost) untestable Apple's API— Live search example
First of all let's make it clear about what a live search is. In a short definition, we could say it is a search which enables continuous input keeping always the most current one as the searched term.
It's totally aligned with some user experience principles as the user will not get blocked during changing his decision regarding what's being searched, even when the request is already running (spoiler).
As it looks like, what really differs a live search is to cancelling the previous request (the running one), replacing it by a new one containing the brand new input term.
Testing it took me some good hours in order to understand how we test a cancel method of a URLSessionDataTask when its indeed being called during some flow or even better (or worst), how its almost inaccessible due some imposed visibility constrains by Apple.
So here its the point about what we want to test.
Seems a bit weird this harmless method being the main actor of the current article but everyone can have their 15 minutes of fame and that guy currently have a lot to share with us.
Let's dive about how to guarantee that each returned task got its own cancel method called, adding some unit tests to it.
First approach — theVolatile
A first and intuitive approach would be basically:
- Access the sut (system under test) session;
- Create some dataTasks;
- Assert the sut's session dataTasks count matches the number of created dataTasks;
- Call the sut method in order to cancel all previous created tasks;
- Assert the sut’s session dataTasks count matches zero since we've cancelled all tasks;
Some concerns about the current approach
As we can observe, the current approach is running tests over real world context (without test doubles). I mean, we have no control about the data being moved over each method, we're just expecting everything works as it should.
Having no control about how data flows over our tests and beyond that tying it to escaping blocks, we are just landing on volatile context, providing all tolls to face flaky tests on a CI however stable it may looks like locally.
Second approach — theDisappointing
We're striving to achieve the beautiful test stability.
An ideia would be creating a test double to a URLSession in order to stubbing session’s getAllTasks() method.
We could create a double to URLSessionDataTask as well and spying dataTask cancel method, in order to guarantee we’ve definitely called it.
Since we have doubles, testing tasks count gets unnecessary due the test double which enables us to assert exactly what we want, the call of cancel method. Look how simpler the test turns out.
Great! Isn't it?
Running the test does not give us any failure over both asserts but actually the test at all. Yes, confusing. Let's check the log.
Task <D6E1DFDB-D7FA-4CCE-B75E-A76223D61FB3>.<5> finished with error [-1001] Error Domain=NSURLErrorDomain Code=-1001 “(null)” UserInfo={_NSURLErrorRelatedURLSessionTaskErrorKey=(“LocalDataTask <D6E1DFDB-D7FA-4CCE-B75E-A76223D61FB3>.<5>”), _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <D6E1DFDB-D7FA-4CCE-B75E-A76223D61FB3>.<5>}
Some points the highlight from it error. The code -1001 represents a timeout error. The error in a complete vision, represents a bad creation of the URLSession, which without a configuration, tends to get the faced scenario.
If the problem is the configuration not being passed, let's just override the URLSession init on our double and pass it. Jumping to the class declaration, we got it:
Seems we have a comment blowing up on our face, making our lifes harder, but we wont give up. As our plan was to override the init, we can conclude our goal got changed 😂
URLSession have public init and not open, making it not overridable, and beyond that, the configuration is a get only property. Tied hands.
Third approach — theSolution
Alright, in one side we have a volatile approach where we faces a bunch of escaping blocks without control of what's being passed in the bowels of the code, on other hand we have control of that but at same time punching a wall of nails due a internal URLSession error.
What would we do?
My suggestion and final implementation to the current problem is to use method swizzling. Basically we'll keep using the "real world" URLSession but replacing its method implementation we want to have control, by a fake one.
YES! We are mixing a bit of the two previous scenarios in order to achieve a controlled context and finally test our dataTask cancel method.
Initially we create an URLSession extension in order to add a static method responsible to swizzle the original and fake implementation.
Method swizzling consists on get each instance method (original and fake), and call a function responsible to exchange both implementations, all of that on runtime. To do that we just need to pass the type of each object holding the implementation and the selectors indicating the respective methods like below.
The FakeURLSession is basically a stub class which provides us control of what's being returned on getAllTasks() method, completing immediately as it is called.
By that, our test will get pretty similar to the second approach, I've just added a new assert before the sut method call in order to check our spied value is indeed changing.
Yaay! We are just on control again, defining what is being returned on getAllTasks() with no need to create custom async control and also matching final values of what we currently have as test goal.
Considerations
There are many subjects on that post which we could get dived deeply but in other point make it longer as it already got. Anyway I have some great articles which would be of your interest.
Special thanks to my amazing team mates Henrique Galo and Giovane Possebon for the great review!
Huge thanks for the reading, hope it could help you in some way ❤
