Android Battery Testing at Microsoft YourPhone
The YourPhone app allows the user to seamlessly connect their PC to their Android device to access a wealth of information from the phone such as messages, photos and notifications. However, keeping a real time connection between such two devices also comes at a cost: Battery consumption.
Especially on mobile devices, it is crucial for an app to have good power performance, as the operating system can flag poorly behaving apps and ask the users to shut them down. For us at YourPhone, having a low power consumption is important, but can also easily impact our product’s reliability and availability. For that reason, we started investigating our battery consumption on Android to understand where we stand and what we can do to monitor our app. We thought it would be a great idea to share our findings and learnings, as this area of work is not very transparent to learn about.
How we got here
We work closely with OEMs that bundle our experience with their devices and regularly perform some type of battery testing to make sure none of the pre-installed apps are heavy battery drainers. For that reason, we decided to set up our own way of recording battery usage to establish a baseline of where we stand today and to be able to catch potential regressions. Our journey started with looking at Android Studio’s profiler to understand how our app works from a battery perspective. The profiler is great at giving an indication of CPU, network and wakelock usage in real-time which helps debug potential battery issues. However, it only provides relative values for these metrics and nothing along the lines of battery capacity drop in mAh which is easier to reason about.
Another tool that the Android framework provides is the Batterystats adb tool in conjunction with the BatteryHistorian tool that visualizes device usage dumps. Those tools turned out to be of great help to us as they provide a lot more information than what is documented. Before we dive into these tools, we should take a step back and understand how battery consumption can be measured.
Measuring battery consumption
Measuring how much power an app uses surprisingly turns out to be a hard problem. In fact, it is not possible to get an exact measurement of an app’s battery consumption. You might now be thinking “but how does my device tell me that app X drains Y% of my battery in my device settings?”. And that’s a totally valid question, but it’s important to know that those statistics in settings use a sophisticated model to approximate each app’s battery usage based on how many system resources are being used. In other words, the system keeps a log of how long an app uses the WiFi, mobile radio, GPS, screen etc. and uses these data points to approximate how much battery consumption that usage amounts to. If you want to learn how this works in detail you can take a look at the BatteryStats implementation in AOSP here (warning it’s 15k loc). The good thing is that this functionality is exposed to the developer through adb with the Batterystats tool, which means that we can make our lives a lot easier and don’t have to make those measurements ourselves.
Using the Batterystats tool
The Android development tools, specifically adb, come with useful debugging tools that we can use to get detailed system information and to dump system logs. One of those tools is the batterystats tool which records system events and uses the previously mentioned BatteryStats model to compute an approximation of each app’s battery usage. When I started looking at the Batterystats and BatteryHistorian tool, I learned that there is a lot more information hidden in the dumped log files than what is documented or exposed through the BatteryHistorian UI. This allowed us to get detailed information on things like network usage as well as battery usage breakdown by device component (e.g. WiFi radio, CPU, Bluetooth, sensors etc.). We quickly came up with a battery plan that revolves around the tool to understand and test our app’s battery life using our existing cross device test framework that runs automated UI tests across a PC and a phone. The idea was simple: wrap our existing tests with battery measurements and then extract and log the data once a test run completes. The implementation of this was straightforward since we just had to issue a few adb commands to the phone and then use some regular expressions to extract relevant data from the log, but there was one more challenge to overcome.
Testing battery while the phone is plugged in
One of our early assumptions was, that we need to test battery performance while the phone is not charging. However, this assumption was not correct given that the battery statistics that we collect are just inferred based on other hardware components instead of being read from the battery. This means that we can charge the device, or even use an emulator to extract battery data which sounds strange at first. There is one caveat though that hindered us from making this discovery early: in order to use the Batterystats tool, the device needs to think that it is discharging, otherwise the data is incomplete and can’t be used. We started out tackling this issue by running our tests over WiFi with adb operating in tcpip mode with the phone being unplugged. That worked great, however made it harder for us automate or deploy our tests somewhere else. For that reason, we did some digging and found out that we can simply use adb to mock the device’s battery state, which in turn allows us to capture battery data using adb. Calling “adb shell dumpsys battery unplug” before recording battery statistics enables us to get a battery usage estimate of devices that are plugged in and even emulators, even though these are technically charging.
Putting it all together
Now that we can easily record battery dumps on any device under any conditions, we can easily hook up battery measurements to our existing cross device tests. All that has to be done for each test run is mocking the battery charging state to make the device say that it’s not plugged in and also resetting the batterystats history in order to only capture data for the desired period of time, i.e. test run. After the tests have executed, we then dump the data and extract relevant information for our app. The batterystats tool dumps a wealth of data of device usage events, such as wake lock acquisition and network usage for each app. In order to extract our app’s usage data, we can simply use the app’s UID to find the relevant battery, network and wakelock usage data. It is also possible to shorten the output of the tool by limiting it to a single app instead of getting the system wide log. This can be done by adding the package name to the adb command, e.g.: “adb shell dumpsys batterystats com.example.packagename”. Using this functionality we can record the following outputs for a cross device photo transfer test run:
With these kinds of measurements, it’s important to remember that the battery usage is just an estimate that is based on what kind of resources our app is using. On the other hand, we cannot get an exact measurement for battery usage by app because it simply does not exist. However, this gives us a good idea of how many resources our app is using, what system components make up for the most battery usage and what battery usage the user would see in their device settings. For us at YourPhone the goal is to use these measurements to benchmark improvements of our transport layer and to catch regressions of resource/battery usage. For that reason, we will start making battery measurements part of our long-haul test suite, which consists of performing the same action repeatedly (e.g. sending notifications). In the future, we will also look at incorporating these measurements into our cross-device UI test suite, however it will be harder to compare results between test runs, since the UI and therefore tests change more frequently than for the long-haul tests.
What about measuring this in production?
Unfortunately, getting this kind of battery usage data at runtime is not supported. The BatteryStats class is only exposed to privileged apps and through adb, which severely limits our options for measuring battery usage. One option is using the BatteryManager class which can be used to query the battery’s charge state and monitor battery usage over time. However, the BatteryManager does not give per-app, but systemwide insights, making it harder to reason about the data. For that reason, we decided that incorporating battery measurements into our test suite is the way to go.
Verdict
After plenty of investigations we were able to find a scalable approach to battery testing for Android. Even if you don’t suspect your app to consume a lot of battery it can still be beneficial to run these kinds of tests to catch potential regressions or excessive network usage. For us, this is just the beginning in our journey with battery testing and we are hoping to learn a lot more about this subject matter in the future.
Appendix: Templates for running battery tests
The following snippet is a simple template for running the battery tests:
The following snippet shows the regular expressions being used for extracting the battery data in C#:
