Testing Android Apps with Firebase Test Lab…

Published in

MindOrks

6 min readDec 20, 2019

What is the Firebase test lab?

For those who have not heard of this — Firebase Test Lab is a cloud-based app-testing infrastructure. With one operation, you can test your Android and iOS applications across a wide variety of devices and device configurations, and see the results — including logs, videos, and screenshots — in the Firebase console.

Have you ever worried about a device-specific crash in production and not having the real device to fix the issue? then Firebase test lab got you. Test Lab exercises your app on devices installed and running in a Google data center, so you can find issues that only occur on specific devices and configurations.

Test Lab is integrated with the Firebase console, Android Studio, and the gcloud command line tool. You can even use it with Continuous Integration (CI) systems.

Let us understand how does it work...

Test Lab uses real, production devices running in a Google data center to test your app. The devices are loaded with new APIs and have customizable locale settings, allowing you to road-test your app on the hardware and configurations it’ll encounter in real-world use.

Test Lab runs Espresso and UI Automator 2.0 tests on Android apps. Write tests using one of those frameworks, then run them through the Firebase console or the gcloud command-line interface. If you’re testing on Android-only, you can even have Test Lab create the tests for you, using automated Robo tests.

In this example, I am using Espresso UI tests to demonstrate how the Firebase test lab works.

Create an Android studio project, Sign in to your Google account and register your app on the firebase console.

Step 1— Register your app:

Provide your package name. Nickname and sha1 are optional.

Step 2— Download Config file:

Download the google-services.json file, and add it to the project

Step 3 — Add Firebase SDK:

classpath 'com.google.gms:google-services:4.3.2' - add this line to project build.gradleapply plugin: 'com.google.gms.google-services' -  add this line to module build.gradle

Step 4 — Set up your testing environment:

testImplementation 'junit:junit:4.12'
androidTestImplementation 'androidx.test.ext:junit:1.1.0'// Espresso
def androidx_test_espresso = "3.1.0"
androidTestImplementation "androidx.test.espresso:espresso-core:$androidx_test_espresso"
androidTestImplementation "androidx.test.espresso:espresso-contrib:$androidx_test_espresso"
implementation "androidx.test.espresso:espresso-idling-resource:$androidx_test_espresso"

// androidx.test
def androidx_test = "1.1.0"
androidTestImplementation "androidx.test:runner:$androidx_test"
androidTestImplementation "androidx.test:core:$androidx_test"
androidTestImplementation "androidx.test.ext:junit-ktx:$androidx_test"

Step 5 — Create two Activities MainActivity and SecondActivity:

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        button_next_activity.setOnClickListener {
            val intent = Intent(this, SecondActivity::class.java)
            startActivity(intent)
        }
    }
}Layout: 
<androidx.constraintlayout.widget.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:id="@+id/main">

    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        android:id="@+id/activity_main_title"
        android:text="@string/text_mainactivity"
        android:textSize="25sp"
        android:textColor="#000"
        android:layout_marginTop="80dp"
        />

    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/button_next_activity"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/activity_main_title"
        android:layout_marginTop="50dp"
        android:text="@string/text_next" />

</androidx.constraintlayout.widget.ConstraintLayout>class SecondActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_second)

        button_back.setOnClickListener {
            onBackPressed()
        }
    }
}Layout:
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:id="@+id/secondary"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <TextView
        android:id="@+id/activity_secondary_title"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="80dp"
        android:text="@string/text_secondactivity"
        android:textColor="#000"
        android:textSize="25sp"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

    <Button
        android:id="@+id/button_back"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="40dp"
        android:text="@string/back"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/activity_secondary_title" />

</androidx.constraintlayout.widget.ConstraintLayout>

Step 6 — Let’s create MainActivityTest:

Let's test two scenarios, launching SecondActivity and navigating back to MainActivity from the SecondActivity.

@RunWith(AndroidJUnit4ClassRunner::class)
class MainActivityTest {

    @Test
    fun test_second_activity() {

        val activity_scenario = ActivityScenario.launch(MainActivity::class.java)

        onView(withId(id.button_next_activity)).perform(click())

        onView(withId(id.secondary)).check(matches(isDisplayed()))

    }

    @Test
    fun test_backPress_toMainActivity() {

        val activityScenario = ActivityScenario.launch(MainActivity::class.java)

        onView(withId(R.id.button_next_activity)).perform(click())

        onView(withId(R.id.secondary)).check(matches(isDisplayed()))

        onView(withId(R.id.button_back)).perform(click())

        onView(withId(R.id.main)).check(matches(isDisplayed()))

    }
}

Step 7 — Let’s create SecondActivityTest:

Launch the SecondActivity and test the visibility of the views.

@RunWith(AndroidJUnit4ClassRunner::class)
class SecondActivityTest {

    @Test
    fun testActivity_inView() {

        val activity_scenario = ActivityScenario.launch(SecondActivity::class.java)

        onView(withId(R.id.secondary))
            .check(matches(isDisplayed()))

    }

    // Check the visibility
    @Test
    fun testVisibility_title_nextButton() {
        val activity_scenario = ActivityScenario.launch(SecondActivity::class.java)

        onView(withId(R.id.activity_secondary_title))
            .check(matches(isDisplayed()))

        onView(withId(R.id.button_back))
            .check(matches(isDisplayed()))
    }

    // Check the text
    @Test
    fun testTitleTextDisplayed() {
        val activity_scenario = ActivityScenario.launch(SecondActivity::class.java)

        onView(withId(R.id.activity_secondary_title))
            .check(matches(withText(R.string.text_secondactivity)))
    }
}

Step 7 — Create a test suite:

A TestSuite is a Composite of Tests. It runs a collection of test cases.

@RunWith(Suite::class)
@Suite.SuiteClasses(
    MainActivityTest::class,
    SecondActivityTest::class
)
class ActivityTestSuite

Run your test in one of the following ways:
To run a single test, open the Project window, and then right-click a test and click Run.
To test all methods in a class, right-click a class or method in the test file and click Run.
To run all tests in a directory, right-click on the directory and select Run tests.

The Android Plugin for Gradle compiles the instrumented test code located in the default directory (src/androidTest/java/), builds a test APK and production APK, installs both APKs on the connected device or emulator, and runs the tests. Android Studio then displays the results of the instrumented test execution in the Run window.

Run your tests with Firebase Test Lab:

There is no charge to test your app with Test Lab within the free daily quota on the Spark plan.

Step 8 — Configure a test matrix and run a test:

Android Studio provides integrated tools that allow you to configure how you want to deploy your tests to Firebase Test Lab. After you have created a Firebase project with Blaze plan billing, you can create a test configuration and run your tests:

Click Run > Edit Configurations from the main menu.
Click Add New Configuration and select Android Tests.

In the Android Test configuration dialog:
Android Studio provides integrated tools that allow you to configure how you want to deploy your tests to Firebase Test Lab. After you have created a Firebase project with Blaze plan billing, you can create a test configuration and run your tests:

Click Run > Edit Configurations from the main menu.
Click Add New Configuration
and select Android Tests.

In the Android Test configuration dialog:

4. Enter or select the details of your test, such as the test name, module type, test type, and test class.

5. From the Target drop-down menu under Deployment Target Options, select Firebase Test Lab Device Matrix.

6. If you are not logged in, click Connect to Google Cloud Platform and allow Android Studio access to your account.

7. Next to Cloud Project, click the settings icon

8. button and select your Firebase project from the list.

Create and configure a test matrix:

9. Next to the Matrix Configuration drop-down list, click Open Dialog

10. Click Add New Configuration (+).

11. In the Name field, enter a name for your new configuration.

12. Select the device(s), Android version(s), locale(s) and screen orientation(s) that you want to test your app with. Firebase Test Lab will test your app against every combination of your selections when generating test results.

Click OK to save your configuration.

13. Click OK in the Run/Debug Configurations dialog to exit.

14. Run your tests by clicking Run