Monitoring App Performance

Illustration by Claudia Sanchez

In this MAD Skills article on monitoring performance I’m guiding you through best practices and provide guidance on tools that can be used to monitor app performance.

Monitoring performance verifies performance moves in the right direction.

App performance can either be monitored in a lab environment, before the app is released and in the field, when users have installed and actively use an app.

The content in this article is also available as recording on YouTube

Lab Monitoring

Performance data gathered here is invaluable, because you can learn how an app performs before it hits users. This helps you limit possible business impact and overall impact on users. Data gathered in the lab can be used as a Go/No-Go signal. If performance metrics worsen between release you can use these as justification to postpone a release or block shipping a feature.

Continuous Integration

You can also have fast turnaround time thanks to continuous integration (CI). There are various CI servers and services available and I won’t compare them here.

By using a CI pipeline you can kick off a benchmark suite every time a commit lands onto a release branch, or maybe even before it gets merged onto your main branch. This way you can compare performance between releases and raise a red flag in case the metrics are above the threshold you decided on.

Make sure to run benchmarks on real devices. Instead of maintaining your own local device pool for testing, you can use a provider that offers testing on physical devices such as Firebase Test Lab.

Firebase Test Lab keeps devices connected and up to date so you don’t have to invest in your local test device setup. You can use the scripting interface to run benchmarks on every build and continuously see results.

Step Fitting

Whether you choose to run benchmarks on local devices or with a cloud provider, spotting regressions can be tricky at times. To automate reporting you have to set thresholds for each metric that you deem important and then compare it to a previously run benchmark. You will see performance fluctuate. Some builds might complete faster and some slower. Spotting whether an increase in runtime for a benchmark is a regression in your code requires you to compare more than two builds.

Within our public AndroidX Continuous Integration pipeline we have faced this problem in the past and came up with a viable solution to it.

Instead of comparing a single build to the previous one, we accumulate a set number of builds and compare builds before and after a change has landed in the codebase.

While this might take a handful of builds to spot a regression, the approach is very reliable and scales well, even with hundreds of libraries and engineers involved.

In his article on fighting regressions with benchmarks, Chris Craik describes exactly how step fitting works in our code base and how you can integrate it into your continuous integration pipeline.

The short version is:

• Write macrobenchmark tests
• Run them on real devices
• Collect and store the output metrics
• Run the step fitting algorithm

You can read all the details about step fitting in this article

Fighting Regressions with Benchmarks in CI

Field Monitoring

You can monitor your app’s performance once your users have started using it, with field monitoring in production.

Android Vitals

The easiest way to monitor production metrics is by looking at the data Android Vitals provides. Android Vitals offers anonymized and aggregated data over your installed user base, along with peer benchmarks so you can compare your app with others. To use Android Vitals, you don’t need to make any changes to your app’s code whatsoever. Simply upload it onto the Google Play Store and we’ll take care of the rest.

Android Vitals provides you with metrics on

• App Startup Time
• Slow and Frozen Frames
• Unresponsive Activities
• Crashes
• and more

You can compare how your app is doing in different device categories such as Android operating system version, available RAM, CPU speed and others.

It’s valuable to periodically check-in on the data provided by Android Vitals to see what could be improved.

Firebase Performance Monitoring

And if you want more details, Firebase Performance Monitoring has you covered.

By adding the gradle plugin to your app, you can see information about cold start times, the rough source of slow and janky frames, and network request durations. All of this data is reported into the Firebase Console.

plugins {
  id(...)
  id("com.google.firebase.firebase-perf") version "$version"
}

And you can get even more specific data using Firebase Performance Monitoring. Begin by adding the library dependency.

dependencies {
  implementation("com.google.firebase:firebase-perf")
}

Then, you can add your own trace points for any code within your app. Here we trace when data is loaded.

Firebase.performance.newTrace(LOAD_DATA).trace {
  loadRemote()
  loadFromCache()
}

By using the Kotlin extension trace function, the call site for loading data can be wrapped. This is a quick way to span multiple method calls.

Alternatively, you can use the AddTrace annotation on a method. This provides you with a way to trace a single method every time it is called and then report it to the Firebase Console.

@AddTrace(name = "loadRemote", enabled = true)
private fun loadRemote() { ... }

We are continuously expanding our suite of Jetpack libraries related to performance.

JankStats

We have released the JankStats library that enables gathering janky frames alongside metadata such as how the user got to the state where a frame had to be dropped. After you add the dependency to your app’s build file, install JankStats into an activity window by calling createAndTrack.

override fun onCreate(savedInstanceState: Bundle?) {
  super.onCreate(savedInstanceState)
    JankStats.createAndTrack(window) { frameData ->
    if (frameData.isJank) {
      logAndAddToReportingQueue()
    }
  }
}

When a frame is considered to be janky by the libraries heuristic, you can log it and add it to your reporting queue.

JankStats holds on to state for you in a PerformanceMetricsState object.

PerformanceMetricsState.getHolderForHierarchy(localView)

Here you can add and remove state in key value pairs.

metricsHolder.state?.putState(
  "Interests:TabState",
  "${tabState.currentIndex}"
)
metricsHolder.state?.removeState(“Interests:TabState”)

This enables you to add metadata such as navigation destinations, which make it easier for you to reproduce how the user reached the janky frame in the first place.

To see more usage of JankStats in action, check out the Now in Android sample, where we have added jank tracking code to key areas such as scrolling through the authors list or the for you page.

That’s all folks

This concludes our second MAD Skills series on Performance. We have shown you important performance metrics and how to inspect, improve and monitor app and library performance around app startup and smooth runtime experiences.

If you haven’t already, go and create a Baseline Profile for your app. It’s a quick way to make your app run faster and smoother.

We’re aware that there’s much more to cover when talking about performance. And while we didn’t have time to cover everything in this series we continue to add more content to our developer guidance and samples.

Also, make sure to ask your questions in the video comments or on Twitter, using #MADPerfQA to get answers directly from the Engineers working on Android Performance in our Q&A session on Sept 1st.

And check out our previous MAD Skills series on performance debugging to get an edge on how you can inspect what’s going on in your code.

Performance Debugging

Go and check out our improved developer documentation, which we have been updating with MAD guidance.

To get more detailed code, check out the samples on GitHub. And take the Macrobenchmarking Codelab or Baseline Profiles Codelab for hands-on guidance through the topics.