Accessing Composables from UiAutomator

How can you use UiAutomator in Jetpack Compose apps?

UiAutomator is a UI testing framework suitable for testing across the system and installed apps. It lets you interact with the visible elements on a screen regardless of what app is in focus.

While many libraries rely on running as part of the app’s process, UiAutomator doesn’t. It enables you to write integration tests, which know nothing about the app’s internals. This is important for generating Baseline Profiles and performance testing (using Jetpack Macrobenchmark) where running a profileable release build is recommended.

In order to deliver reliable results, the testing framework needs to interact with the app but without manipulating the app directly. And UiAutomator does just that, by delivering input events in a way a regular user would, but more consistent.

UiAutomator basics

UiAutomator has several ways of accessing UI elements on screen depending on an element type. You can use the By selector class for accessing elements. You can use By.text() to access elements by text label, By.desc() to access by contentDescription, by element flags such as By.scrollable(), By.checkable(), By.clickable(), etc; and By.res() to access an element by its resource-id android:id="@+id/some_id".

In the View system, you usually set up an identifier to enable access and set properties of that View. In this case accessing an element is not a problem, because you can use By.res(packageName, "some_id") to get the handle on that View and then interact with it.

Jetpack Compose renders UI differently than the View system. Compose has a declarative way of describing UI, so it doesn’t require resource identifiers (android:id). While it's very convenient for developers, it can be problematic for UiAutomator to access elements that don’t have unique identifiers such as text labels or an element flag. This is especially important for layout components, such as Row, Column, LazyColumn, and others.

Technically, it’s possible to use the contentDescription to get access to a composable, but use it only when it makes sense for accessibility purposes. The contentDescription parameter is used by the accessibility framework and if you add tags that aren't important to humans, you effectively make your app harder to use for those who rely on accessibility.

Instead, with Jetpack Compose you can leverage Modifier.testTag(). This is not enabled by default, so let’s see how you can enable it.

Enable UiAutomator interoperability with Jetpack Compose

First, you need to enable testTagsAsResourceId in the composable hierarchy you want to test. This flag will enable converting the testTag to resource identifiers for all nested composables. If you have a single Activity Compose project, you can enable it only once close to the root of the composable tree. This will ensure all of the nested composables with Modifier.testTag are accessible from the UiAutomator.

Note: This flag is available in Jetpack Compose 1.2.0 or newer. If you use compose-bom, any version will contain this feature.

In the Now in Android sample, we modified the Scaffold, which is part of the NiaApp composable. This composable is the root composable (except for NiaTheme, which is not capable of setting any Modifier). You can add the Modifier.semantics with testTagsAsResourceId = true as shown in the following snippet:

/* Copyright 2022 Google LLC. 
   SPDX-License-Identifier: Apache-2.0 */

Scaffold(
  modifier = Modifier.semantics {
    testTagsAsResourceId = true
  },
  // ...
)

Once you have that, you can use Modifier.testTag("identifier") anywhere in the Composables hierarchy and the identifiers will be propagated to the UiAutomator as a resource-id.

In the ForYouScreen composable, let’s add the Modifier.testTag("forYou:feed") to the LazyVerticalGrid. The parameter name is arbitrary, you don't need to follow the same pattern we selected for Now in android.

/* Copyright 2022 Google LLC. 
   SPDX-License-Identifier: Apache-2.0 */

LazyVerticalGrid(
  modifier = modifier
   .fillMaxSize()
   .testTag("forYou:feed"),
  // ...
)

You can now access the LazyVerticalGrid from the Ui tests without the need to sacrifice content description for testing. From the tests, you can use By.res("forYou:feed") selector.

In our case, we use the UiAutomator for benchmarking and generating Baseline Profiles.

For Baseline Profile generator, you write an instrumentation test like in this following snippet.

/* Copyright 2022 Google LLC. 
   SPDX-License-Identifier: Apache-2.0 */

class BaselineProfileGenerator {
  @get:Rule 
  val rule = BaselineProfileRule()

  @Test
  fun generate() {
    rule.collectBaselineProfile(PACKAGE_NAME) {
      // This block defines the app's critical user journey. 
      // Here we are interested in optimizing for app startup. 
      pressHome()
      startActivityAndWait()
    }
  }
}

This test will start the default activity several times to generate a Baseline Profile.

But you can even go beyond just app startup optimization. You can optimize the runtime performance of the feed that is loaded on the first screen. So you wait for and find the feed list by using By.res("forYou:feed") selector and do some interactions with it.

/* Copyright 2022 Google LLC. 
   SPDX-License-Identifier: Apache-2.0 */

class BaselineProfileGenerator {
  @get:Rule 
  val rule = BaselineProfileRule()

  @Test
  fun generate() {
    rule.collectBaselineProfile(PACKAGE_NAME) {
      // This block defines the app's critical user journey. 
      // Here we are interested in optimizing for app startup. 
      pressHome()
      startActivityAndWait()
      
      // Wait until content is asynchronously loaded.
      // We find element with resource-id "forYou:feed", which equals to Modifier.testTag("forYou:feed")
      device.wait(Until.hasObject(By.res("forYou:feed")), 5_000)
      val feedList = device.findObject(By.res("forYou:feed"))

      // Set some margin from the sides to prevent triggering system navigation
      feedList.setGestureMargin(device.displayWidth / 5)

      // Fling the feed
      feedList.fling(Direction.DOWN)
      device.waitForIdle()
      feedList.fling(Direction.UP)            
    }
  }
}

Be aware! There’s a difference in using By.res(packageName, "identifier") and By.res("identifier"). The former will be searching for @packageName:id/identifier in the UI hierarchy, while the latter just for identifier, which is required for the Modifier.testTag.

In our example, if you’d use By.res("com.google.samples.apps.nowinandroid", "forYou:feed"), it would result in the UiAutomator trying to find an UI element with @com.google.samples.apps.nowinandroid:id/forYou:feed resource-id, but it doesn’t exist, so the test fails with java.lang.NullPointerException. The correct way, without the package name — By.res("forYou:feed"), will resolve to forYou:feed resource-id, which is correctly found on screen.

What’s next

After reading this article you have seen that enabling UiAutomator interoperability with Jetpack Compose is easy and you can leave the content description for accessibility purposes.

To dig into a more complete sample, check the Now in Android benchmarks module that generates Baseline Profiles and measures performance. The performance-samples Github repository also uses this interoperability while also showing other performance samples. For more information about Baseline Profiles, check the documentation, or, if you like a more hands on approach, check our Baseline Profiles codelab. Also, you may check the new updates for UiAutomator.

Next task — writing the custom Baseline Profiles generator and getting the performance improvements.