Automation techniques — (level-1) verifications

GUI testing is flaky, difficult to maintain, slow to run but often very necessary. Especially in context of mobile app automation, where a large part of testing has to be done through GUI.

One of my favorite techniques to alleviate the pain in these situations is to do verifications at a non-GUI, a layer beneath kind of tests. Our actual test action will take place normally but the verification happens at a lower level. These verifications can be like a API call or a method call or some other sort of action that provides the same value as the GUI verification.

A properly architected test contains the following parts

1. Setup — get the test to the state where you can do your action
2. Action — perform the action that you want to test
3. Verify — has the test passed or failed
4. Cleanup — undo any state changes setup or action has done

On closer inspection, even for a GUI level test, only step 2 — action has to be done at that GUI level, some (most) of these parts can be delegated to a lower level verification. Let’s take a concrete real example — for the test statement

Verify that a user can add a domain to his favorites

If I were to do break this test into the above 4 parts, they would look something like

1. Setup — login, finish on-boarding, go the domain screen where favoriting is possible
2. Action — click on the favorite button
3. Verify — Verify that the domain is really favorited
4. Cleanup — Unfavorite the domain and exit the app

In this case, the test case would still be very valid if I were somehow able to do step 1, 3 & 4 through API layer or other techniques. Say, we have a REST call of this nature
http://host/12345_domainid/details and it gives us back a JSON blob with all its state including the favorite, that’s the perfect chance for us to go for the API layer validation

Level minus one (level down) verifications will give us the following benefits

• They will be fast — we are circumventing the UI, API actions are always faster than UI
• They will be reliable — again no flakiness of the UI
• They can be controlled through programmtically through objects

Obviously no matter how hard you try some of these steps cannot be circumvented and in those cases you will have to come back to the GUI layer but in 8/10 cases, if you analyze the situation and get proper testhooks in place it will be possible to do these tests at a lower layer.

In my next post I will explain a practical application of this priciple, where we enabled such level-1 verifications.