Adapting Mobile Application Testing to Android Automotive OS
Written by: Luna Ticeu, Agile Test Engineer, TribalScale
📫 Subscribe to receive our content here.
💬 Have any questions about our Android Automotive capabilities? Click here to chat with one of our experts!
Check out our previous two articles where we:
In this article I’ll be sharing how I adapted my testing techniques for AAOS and the automotive world.
As an Agile Test Engineer at TribalScale, I’ve worked on a lot of different projects that have exposed me to new cutting-edge technology. This is a constant challenge and gives me the opportunity to learn a lot which I love.
Recently I have the chance to work with Android Automotive OS (AAOS)—tech that was entirely new to me, however I couldn’t wait to get my hands on it. Although I have been working in Software Quality for over 7 years, this was my first time testing anything related to Automotive technology. From my years of experience with mobile testing, I was able to use this basis of knowledge for AAOS.
Get To Know the Technology
Before I start any project, I research the technology in order to get an understanding of what I can expect from the system. From there I’ll have better insight into where I should focus my testing and what type of testing I should perform.
In my research I came across this important info on Android’s website—“With Android Automotive OS, users install your app directly onto the car instead of their phones.” Essentially, AAOS uses Android technology on a head unit running directly on the in-vehicle hardware.
After absorbing this, I decided that the best approach would be for me to personalize the mobile testing methodology. Why is it important? Sometimes when testing something you’ve never worked with, you don’t know how to approach it. But by relating it to something you already know, you can adapt your pre-existing knowledge to find a solution.
Map the Risks
One of the most important things for a QA is to know your software risks. Risk analysis is a good practice to identify the critical areas of the software and helps us decide where to focus the necessary effort during the testing phase.
The risks I uncovered were:
- The software may not perform functions as specified.
- The software may not perform the intended functions according to the user’s needs.
- The architecture may not adequately support some non-functional requirements.
- Response times may be inadequate.
- User difficulty touching the screen.
- Adding voice commands.
- Addition of the car’s physical state to the parked and unparked test complexity.
Personalized Test Strategy
A good mobile testing strategy typically focuses on testing many different devices through a cross-platform or maximum coverage approach. We then narrow the scope to a representative selection of devices and operating systems used by the majority of customers in the target market. Additionally, we can include covering all operating system versions, devices, manufacturers, internet providers, and network types.
But that is not the case with Android Automotive OS, being such a new technology, many of these challenges are not applied because the head unit has a specific screen size, a limit of supported operating systems, and defined hardware technology.
Using the single-platform approach is the best for AAOS, by limiting the scope to a single device type, OS version, internet provider, and network type you can focus on targeting audiences and providing the best experience possible.
Find the Supported App Categories
Understanding what types of applications are supported, and what actions and features can be expected is also a very important input for the quality of the application.
Android Automotive OS supports the following types:
- Media apps — using web browsers to play music, radio, audiobooks, and other audio content in the car.
- Messaging apps—receive incoming notifications, read messages aloud using text-to-speech, and send replies via voice input in the car.
- Navigation, parking, and charging apps—helping users get where they want to go, find a place to park or locate the nearest charging station.
- Video apps—viewing streaming videos while the car is parked.
Types of Tests That Can Be Applied
To define which types of tests best fit the Android Automotive OS context, it is necessary to set the test objectives first.
The goals I prioritized were:
- Evaluate the functional quality characteristics.
- Assess non-functional quality characteristics.
- Assess the system architecture.
- Evaluate the effects of the changes (confirmation of the correction of defects).
- Look for unintended changes in behaviour as a result of changes to the software or environment (regression testing).
Usually, it is not enough to just test if the application works correctly with the expected resources and entries. If a resource is missing, you’ll need to test that the app still works as expected.
That is why using different types of testing is so important. The testing types I executed were:
Device feature test
In-vehicle infotainment systems can use different types of features available in the head unit. For example, there are different ways to navigate and turn off the system, and there are different types of keyboards (physical and virtual), Bluetooth and microphone.
Device input sensors test
Devices can receive a variety of types of input. For example, buttons, real-time location, temperature, humidity or touchless inputs.
Testing various input methods
Testing various input methods verifies that the default keyboard displays with the proper layout and keys, single or multi-finger taps, gestures, or other inputs, and ignores any gestures or inputs that are not supported.
Interrupt test
The most common types of device outages include general messages and notifications. User-initiated interruptions result from actions such as switching applications or setting the device to sleep while the application is running.
Interrupt tests verify that the system handles all interrupts with no negative impact on application behaviour and that the application continues to function correctly, preserving its state, data, and sessions.
Testing access permissions
Some apps need access to folders and sensors to work as expected. When access is denied on installation or changed after installation, it can affect application behaviour.
Tests for access permissions validate that only requests are made for features relevant to the application’s functionality as the system handles reduced permissions, and ensure that there is no unexplained failure.
Notification test
There are several mechanisms used by the operating system to display notifications. The testing phase should verify that the handling of incoming notifications is correct when the app is in the foreground or background, that the notifications allow direct interaction with the app’s content, and that the notifications allow direct access to the app.
Testing system-provided user preferences
Any settings preferences provided by users in the operating system should be tested. This can generate a negative experience for the users if a certain preference setting is not respected.
Testing interoperability and coexistence with others
It is quite common for applications to interact with each other when installed on a device. Testing should verify that the data transfer between the system under test and the application being used is correct, that there is no damage to stored user data, and that there is no conflicting behaviour between applications which is very important.
Find the Best Test Environment
Now that we have defined what to test and how we will test, all that remains is to answer the question “Where to test?”.
As I said before, Android Automotive OS runs on a specific head unit. An automotive head unit is a part presenting a unified hardware interface for the device, consisting of screens, buttons, and device controls for several incorporated records and amusement functions.
When we talk about testing environments, we have a few options on where to run it, on real devices or emulators and simulators.
When I wrote this article, I couldn’t get my hands on a real device and I couldn’t find any simulator that was compatible with my testing needs. So my only option was to use an emulator.
Final Considerations
Once I understood the differences and similarities between testing a mobile application and an Android Automotive OS application, I felt more confident in being able to add quality from the beginning to the end of the project lifecycle.
By adopting some of the processes I already knew and understanding the limitations of the new, I could focus my tests on the areas of greatest risks, project needs, and achieving my testing goals faster.
Check out the next part of this blog series, where we share how we designed the AccuWeather app for Android Automotive OS.
References
Luna is an Agile Test Engineer at TribalScale with experience in testing involving Web, Desktop, Mobile and API applications. She is passionate about software testing, loves exploring new technologies and thinking about how to break them. Outside work, she spends her time reading and has a goal to explore Calgary this summer.
TribalScale is a global innovation firm that helps enterprises adapt and thrive in the digital era. We transform teams and processes, build best-in-class digital products, and create disruptive startups. Learn more about us on our website. Connect with us on Twitter, LinkedIn & Facebook!
