White-box testing with Appium Espresso Driver

At Badoo, we have techies who strongly believe in contributing to the open source community. This article is about how I contributed a very useful feature called ‘backdoor’ to Appium which solves lots of problems related to Android UI automation. There are a few examples of it in this article. The good news is that this feature has already been accepted, merged and released by Appium! So, it’s now available to Appium users out of the box.

An avid ‘masala chai’ drinker, I always carry a thermal tea flask like one below with me.

Many times, desperate for a sip of tea, I misjudge the temperature and burn my tongue. It is unfortunate that such a mishap happens to a tester because I should, of course, first test the tea temperature and then drink it! However, I have no information about the tea because it’s completely hidden inside the mug. Sometimes, I don’t even know if there is any tea left in the mug at all because this mug is, effectively, a black box. I wish I was able to ask the tea, ‘Are you too hot?’

Actually, this problem is not just a problem with my tea mug, but also with the mobile apps for which I write automated tests. There are times when I need to find out about the internal state of the app, or value of some property of the element on screen but I can’t because the automation tools usually only support Black-box testing. One such tool which we use at Badoo for android automation is Appium with uiautomator2 driver.

Some recent Android testing tools like Espresso allow White-box testing where internal app methods are accessible to the automated tests. The Appium community recently created an espresso backend using appium-espresso-driver which made it possible for us to make grey-box testing work with Appium. I was fortunate to work on the appium-espresso-driver codebase and contribute mobile:backdoor command to allow white box testing with Appium. The ‘mobile:backdoor’ command can be used to invoke methods defined on an activity, the application class or a specific UI element. In this article, I will explain a few tricky cases which can be automated by using ‘mobile:backdoor’ on a UI element.

Before I jump into examples, if you are looking for some that are ready-cooked, click here for github-repo of backdoor ones.

An Example with Rating Bar

Suppose you have to automate a case where the user gives a 4.5-star rating in below screen. Here, RatingBar is one element with no access to individual stars inside. Did you just think you could find the coordinates of the 5th star and then tap slightly left of the centre? But how uncool is that? What if the device size were to change?

There is a sane way to do this using mobile-command “mobile: backdoor” in Appium. All we need to do is to check public methods available on android.widget.RatingBar class. It is easy to spot setRating(float) method available in the list. We can invoke this method from the test code as below:

Note that this example is in Ruby, but you can do the same in Java or any other Appium client of your choice.

e = @driver.find_element(id: 'ratingbar_id')
@driver.execute_script("mobile: backdoor",
                     {
                         target: "element",
                         elementId: e.ref,
                         methods:
                             [
                                 {
                                     name: "setRating",
                                     args: [{type: 'float', value:
"4.5"}]
                                 }
                             ]
                     })

Here is an explanation of the parameters:

• “mobile: backdoor” → This is the name of command
• target: “element” → This implies that we want to call a method defined on Android Element. There can be other targets like activity and application but these are beyond the scope of this article.
• elementId: e.ref → For target type element, we need to tell internal element Id on which we need to invoke a method
• methods → This is an array of method chain we want to call. In our case, we have just one method with the name as “setRating” to be called. This method takes an argument of type: ‘float’ and value as “4.5” (number of stars we want to set for our Rating bar).

Once you invoke this script, you will see the Rating gets set to 4.5 immediately.

Another example: Getting Text Color using Appium

Suppose you are testing an app which is design heavy (e.g. a blogging app) and you need to test some properties of texts on UI. There are a lot of questions on Appium forums about how to get text color. To date, there has been no way to do this. But now, it can be done using element backdoors. Here is a small code example of how to do this:

element = @driver.find_element({id: 'message'})
color = @driver.execute_script("mobile: backdoor",
                             {
                                 target: "element",
                                 elementId: element.ref,
                                 methods: [
                                     {name: "getCurrentTextColor"}
                                 ]
                             })

We get hex color value back from the above script.

Clicking a link inside multiline text

Now, that we understand the above examples, let’s have a look at a slightly more complex example.

The goal here is to tap on a link inside a long text (or any given word in a long multiline textview). Say, as shown in the image below, we need to click the link “Change?” in the long text.

If you thought of finding the coordinates of the word ‘Change?’ and clicking on it, it may work but the problem is how to reliably find the coordinates which will work on different devices and also when screen orientation changes.

Here is how we can request the coordinates of a word in a multiline string textview:

def tap_subtext_in_text(subtext, locator)
 text_view = @driver.find_element({id: locator})
 full_text = text_view.text
 link_index = full_text.index(subtext) + subtext.size / 2
 top_x = text_view.location.x
 top_y = text_view.location.y
 ref = text_view.ref
left_padding = @driver.execute_script("mobile: backdoor", {target: "element", elementId: ref, methods: [{name: "getCompoundPaddingLeft"}]})
# x coordinate of word (link)
 hor = @driver.execute_script("mobile: backdoor", {target: "element", elementId: ref, methods: [{name: "getLayout"}, {name: "getPrimaryHorizontal", args: [{type: "int", value: link_index}]}]})
# line number at which word(link) exists
 line = @driver.execute_script("mobile: backdoor", {target: "element", elementId: ref, methods: [{name: "getLayout"}, {name: "getLineForOffset", args: [{type: "int", value: link_index}]}]})
# y coordinate of word (link)
 ver = @driver.execute_script("mobile: backdoor", {target: "element", elementId: ref, methods: [{name: "getLayout"}, {name: "getLineBaseline", args: [{type: "int", value: line}]}]})
x = hor + top_x + left_padding
 y = ver + top_y
 touch_point(x, y)
end
def touch_point(hor, ver)
 selenium_driver = @driver.driver
 f1 = selenium_driver.action.add_pointer_input(:touch, 'finger1')
 f1.create_pointer_move(duration: 0, x: hor, y: ver, origin: ::Selenium::WebDriver::Interactions::PointerMove::VIEWPORT)
 f1.create_pointer_down(:left)
 f1.create_pause(0.2)
 f1.create_pointer_up(:left)
 selenium_driver.perform_actions [f1]
end

This would work for any device of any screen size. Such a method in your automation framework will make your tests very robust.

That’s all there is to it! The above examples just scratch the surface and there will be a lot of other use cases of ‘mobile:backdoor’ with element as target. In the next article, I will look at other two targets activity and application and share some interesting use cases. If you are curious to know how this works under the bonnet, or you have any questions, or you have found a cool use-case of executeScript(“mobile: backdoor”) command in your automation, do include details in the comments below.

Also, stay tuned for more articles.