In my brief experience with Android developers and community so far I have seen lot of people talking about automated testing and its importance, but a very few people get the chance to actually write extensive tests for their code. They hardly ever have time for it. I also happen to be one of them. Although I have heard, read and talked about it a lot, I have very limited hands on experience.
In one of the apps I am currently working on, team finally decided to start writing tests for good. We were first aiming to build a smoke test-suite for complete app so we started with UI testing with Espresso. We also use Koin as DI framework in this project.
Honestly, I found that testing on Android is a mess for beginners at-least. In this article I am going to point out some of the pitfalls and mistakes that we came across.
There are literally tons and tons of google libraries available which are available for testing. Official AndroidX testing guide lists more than 10 libraries for doing just UI tests. These are the ones that you add as
androidTestImplementation in your
build.gradle similary there are other bunch of libraries for JVM unit tests. Not just these, in lot of tutorials, code samples and Stackoverflow posts you will find older non AndroidX version of these.
I would recommend to not include all these at once in your project. Just start with one and then include the ones that you need, so that, you exactly know what you are including and for what. This just helps in building understanding step by step.
I required following libs in order to make my bare minimum sample work
2. Test/Mock/Fake Application
You need a mock application class as an entry point for UI tests. You can initialise your dependency graph here. Instead of your actual
Application class this will instantiated with your tests. You will also need a custom
Runner instance where you will instantiate Application. Here is my
Here is my
You will then also need to specify this test runner in your
3. Mockk vs Mockito
I started with Mockito for mocking dependencies. Then A few people recommended Mockk over
Mockito. I tried it and I liked it initially. One of the great thing about it is that it doesn't require you to declare your
Kotlin classes and methods as
open. Also I found that there were fewer issues to deal with when using
Mockk then with
But then I tested my app on API 27, tests that were running fine on API 28 and 29 suddenly started breaking. Turns out
Mockk does that
open magic only for devices with API level >= 28. This was a deal breaker!
At this point we decided to go back to
Mockito as its a established library, some of the test libraries like
koin-test were already using it under the hood and we will have one less library to deal with.
With our limited knowledge of
Mockk this is not a final decision. If in future we feel that Mockk is more suitable option for Kotlin then we may decide to migrate.
@OpenForTesting to Rescue
Yigit Boyar wrote this very useful OpenForTesting annotation which can be used to make all classes and methods open just for testing. Here is a great step by step tutorial on how to set this up in your project. This worked great and now our tests started passing on API < 28.
Remember even if any of your class is
open in your codebase you still need to annotate it with
@OpenForTesting(or whatever you choose to name it) in order to be able to mock its methods. Remember, that in
Kotlin all methods in a class are closed by default. We spent hours on this because somehow, we managed to ignore this fact.
Test rules are great way of reusing code across your tests. Here is a great article that explains test rules in detail. ActivityTestRule is one such test rule provided by good folks at Android. It takes care of managing an
Activity environment for your test.
My bad that I didn’t read/interpret documents properly and got an issue where I wasted a full day. My problem was that this rule was launching activity even before I loaded my dependencies with
Koin and, as a result was getting
NoBeanDefFoundException. More on this issue on this Stackoverflow post. (Special thanks to basilisk for solution :))
All was good till this point and now my dependencies were getting loaded properly. I went on and wrote my first line in
You would think what can go wrong with that, right? Well first exception that I encountered here was
PerformException: Error performing ‘type text(Hello World!)’ on view ‘Animations or transitions are enabled on the target device.
As per official Espresso docs device animations should be turned off. More on this exception in next section.
After I turned off the animations I fired up my test again only to find this exception
Caused by: androidx.test.espresso.InjectEventSecurityException: java.lang.SecurityException: Injecting to another application requires INJECT_EVENTS permission
Well, I just tried to write a string in a `EditText` didn’t I? Turns out that Android 10 running on my phone tries to autofill the given editText. Now this auto-fill is important from user experience perspective and I cannot disable it in XML. Thanks to `ActivityRule` I was able to do this
Random Issues ?!?
Most surprising thing about this endeavour were random issues, some of these issues appear and disappear without even a code change. Cleaning the project, connecting disconnecting device seem to affect these but there is no clear pattern.
For example I was almost always getting an exception complaining that
ViewModel.clear() method wasn't mocked or definition not found. This method is final and cannot be mocked. Workaround is to add
ActivityRule.finishActivity() call in
@After method. Surprisingly, I am not able to reproduce this at the time of writing.
There are time when a tests executes twice, it passes first time and then fails the second time. On
Stackoverflow some people were suggesting that this might be happening because test are being run once as a part of suite and then individually. I am not fully convinced here, however, cleaning the build and re-running the test seem to resolve this issue.
Turn off Animation
Espresso tests require that device animations should be turned off. However, I have observed that if you pass
ActivityTestRule this is no more a requirement.
There is a gradle flag which allows you to disable device animations.
Well, this doesn’t work or at-least not reliably in my experience. Don’t waste your time on this like I did.
As of now we have started writing our tests without disabling them. But, if this a problem in future we can consider deploying some custom solution like this.
As you can see it was pretty daunting to setup UI unit testing in our Android app. I hope this was one time annoyance. I have tried to compile all the issues that me and my team faced so that you as a reader know what you are getting into. Here is the source code of project I was working with.
Please feel free to drop your feedback and suggestions.