Fixing Broken Tests in Android Studio After Upgrading to AndroidX

At Apartment List, we believe we can help every renter find a home they love. Our engineering team is focused on creating experiences that make the rental search process more simple. One of our company core values is to “Find a way.” This is one story narrating how we found a way to solve a difficult problem we faced.

The Android Support Library was introduced by Google as a way to provide newer features for older versions of Android. What initially started as one library evolved into a set of libraries that were essential to Android development. With the release of Android 9.0 and Android Jetpack, these libraries have moved into a new namespace called AndroidX. At Apartment List, we try to stay current with our libraries. When the stable version of AndroidX was released, we decided to upgrade. Unfortunately, the transition was not smooth for a variety of reasons. The issues we encountered while updating our code are similar to those mentioned in other blogs. This post isn’t about problems with the AndroidX migration tool or the many other issues that have been covered elsewhere. This post is about an issue we encountered after upgrading that appears to be very rare, or at least not written about much yet.

After the upgrade to AndroidX and Android Studio 3.3, everything functioned great until we started building new features. The feature work started with building a layout and model, then adding business logic. After we started to add business logic, the next step was to write unit tests to verify our code worked correctly. This is where I encountered the problem: attempting to run tests in Android Studio via the green play button in the IDE gutter resulted in the UI displaying text reading: “Nothing here.”

The next step was to check if our tests ran in terminal. Since the CI build did not fail, there was no reason to think running tests from terminal would fail. As expected, the entire test suite ran without an issue. Adding the new test caused the test suite to fail, but it was basically impossible to debug from the terminal. Our only option was to write perfect tests the first time.

Once it was clear that we had a problem, we started looking for a solution. Of course, the first step was to try the usual suspects: clean, build, rebuild, restart IDE, restart computer. None of those made a difference. This was a hint that it wouldn’t be an easy fix.

If the IDE tests are broken because of AndroidX or Android Studio 3.3, then there must be a decent number of people out there with the same issue, right? Not exactly. It was surprisingly hard to find information about this problem. One of the posts on this topic said the issue did not happen in Android Studio 3.4 preview. I tried upgrading to that version, but still had the same issue.

I eventually found a GitHub thread about the exact issue. Better yet, there was also a solution! The issue appeared to be related to the Spek testing framework plugin for Android Studio. There was a link to the latest version of the plugin that was not yet available through the Android Studio plugin manager. After installing the plugin we held our breath while the IDE restarted, but still could not run the tests.

I verified that my machine was running the same version of everything mentioned in the GitHub thread. I asked a coworker whose tests had been failing but were now working to export his Android Studio settings to me. Neither one worked. Later it turned out the coworkers working tests were a fluke. Next time he tried to run them he also faced the dreaded “Nothing here” message. In the meantime, work continued on other parts of the feature with the hope that returning to the test issue later with a clear mind could reveal something we missed before. Unfortunately, there was no breakthrough upon returning to the problem a few days later. I was stuck in a cycle of frustration.

A last ditch attempt at fixing the problem was to incorporate all solutions I came across for any issues that appeared to be even remotely related. Surprisingly, this actually worked! I finished implementing the feature while living in fear that the tests would fail to run at any minute, but they kept working. The changes are summarized below:

• Update Kotlin Gradle Plugin
• Update Gradle Tools version
• Update Kotlin Mockito
• Remove exclusion of group JUnit Platform
• Update RecyclerView version
• Exclude two groups from Facebook SDK
• Update Gradle version

classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:1.3.20"
classpath "io.fabric.tools:gradle:1.27.0"
testImplementation "com.nhaarman:mockito-kotlin-kt1.1:1.6.0"
-- exclude group: "org.junit.platform" (removed this line)
implementation "androidx.recyclerview:recyclerview:1.1.0-alpha02"
// Facebook
implementation ("com.facebook.android:facebook-android-sdk:$facebook_version") {
    exclude group: 'com.parse.bolts'
    exclude group: 'com.google.zxing'
}
distributionUrl=https\://services.gradle.org/distributions/gradle-5.2-rc-1-all.zip

Once everything was complete, I went back to remove unnecessary changes in the Gradle file. At each step, I was scared that the tests would break. Fortunately, this part was a lot easier than the initial fix. I removed each change one by one and ran the tests. If the tests didn’t run, I added back the specific change and moved on to the next change. In the end, the minimum changes required to make the tests run in the IDE were:

• Update Kotlin Gradle Plugin
• Update Gradle Tools version
• Update Kotlin Mockito
• Update Gradle version
• Not committed initially because it was a Release Candidate, but was committed after next stable release

We’re still not sure why we were one of the few groups that encountered this problem, but I’m glad to say that we found a way and finally got it working.