Testing Android voice apps automatically

Let’s review AndroidViewClient/culebra concertina mode features, compare them with monkey, and see how we can use those features to test voice based UIs or Alexa Skills

You may have user monkey, the UI/Application exerciser included in Android SDK. If you didn’t, let me tell you that it is a program that runs on your emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as some system-level events. You can use the monkey to stress-test applications that you are developing, in a random yet repeatable manner.
The keyword here is repeatable. There are some ways of turning monkey more repeatable, like specifying the random seed in the command line, but still, you may receive unexpected results.
Try rebooting your device or emulator and running this a couple of times.

adb shell monkey -s 1234 -vv -p com.android.calculator 100

chances of ending in entirely different places are high.

The alternative is to script `monkey`, but the only way is by using its pseudo-language, and the options are insufficient. Without variables or flow control, the scripting possibilities are entirely limited.

When some time ago I wrote the book “Android Application Testing Guide” (ISBN:9781849513500) before I even started AndroidViewClient/Culebra I dedicated the last part of a chapter to this technique (See Chapter 5).

I will reproduce a similar example here so we can get a better idea of the feasibilities.

Start monkey server

adb shell monkey -p com.android.calculator2 — port 1080 &

forward the port

adb forward tcp:1080 tcp:1080

send a script using netcat

nc localhost 1080 <<EOF
tap 500 500
type 123
press DEL

The main idea was to start monkey in server mode and send scripts. Needless to say, that approach becomes easily unmanageable when the complexity or length of the tests increase.

So, here we are, several years later, describing a better solution. Notice that I didn’t say introducing because it was also added some time ago, but it wasn’t until recently, after answering a question in StackOverflow, that it became more flexible and adaptable to broader user needs.
If you are familiar with AndroidViewClient/Culebra, you may have guessed that I’m talking about “concertina mode.” If you aren’t, don’t worry, as I’m going to explain it here in a detailed way.
Named after the “concertina movement” occurring in snakes and other legless organisms that consists of gripping or anchoring with portions of the body while pulling/pushing other sections in the direction of movement, this mode allows Culebra to move across the UI.

Unlike monkey, which sends pseudo-random events, Culebra concertina mode analyzes the content of the screen and randomly selects a suitable event or action for the also randomly chosen target, usually a View.
Also, unlike monkey, the generated actions and their parameters are saved in a python script that can be later executed as many times as needed.

For example, if the randomly selected view is an EditText, Culebra enters some random text. However, if the EditText turns to be a password, it sends random passwords, that is, instead of just typing a standard sentence it selected sample passwords from a list like the infamous “querty” or “123456”.

Furthermore, if it detects the EditText is expecting an email address, by inspecting the id, the hint, the content description or anything else that may suggest it is an email address, it sends randomly chosen values like user@example.com. Of course, other Views, receive different treatment, as Buttons clicked, Scrollable scrolled, etc.

Additionally, if the content description suggests the application may be waiting for you to talk, like ‘Tap to Speak,’ ‘Ask Alexa’ or ‘Voice Search,’ Culebra will actually speak random text to the app (only available on Linux and OSX).

This feature can be used to create Alexa Skill tests using the Android application.
To demonstrate it we will use a simple utterance like

Alexa, what is the price for Bitcoin?

And we will analyze the response. To achieve this, we get the content of a View that matches “1 Bitcoin is worth.*” as we don’t know the price in advance and we don’t want a test that fails if the Bitcoin price changes, as this is not what we are testing.

The code for this simple script leveraging on the features that make Culebra concertina mode possible is