Test Automation Demystified, Part 5: Codeless UI Test Automation

9 min readJul 10, 2019

What does codeless mean? Obviously, it means to code less, write less code. Even software developers use codeless techniques. They do not program everything. They actively use code generators, compilers, visual composition tools, models. So codeless test automation is not something new. It has deep roots in software development.

Why we need codeless? For two main reasons. Firstly, it should speed up the testing process, allow to produce more results in less time. Secondly, it should lower the learning curve and demand less skills from a user. This, second, aspect is very important. These days we use so many software systems that we simply have very small amount of time to spend on learning, and it is very hard to keep all the necessary details in active memory. A software system should guide a user wherever possible.

Software testers come from very different background, some from manual testing, some are domain experts, some are developers. All of them should benefit from codeless features of a software testing tool.

I would compare codeless approach with electric bikes. Having an accumulator on your bike allows to get further, with less efforts and requires less skills. It allows everyone to enjoy mountain nature without ability to produce 1000 watts of power like a pro cyclist.

Electric Bicycle is licensed under CC BY-SA 4.0

How to write less code in UI test automation? What are common ways of achieving the goal?

1. Still write code using one of high-level programming languages and use software libraries (like Selenium and Appium) that help simplifying some tasks.

2. Use a tool providing keyword-driven testing features.

First option is still good old programming. Feature set of the second option greatly depends on a tool used.

Keywords

What is a keyword? It is a name for an action to perform over the application being tested. A keyword may use input data like UI element to interact with and text to enter.

Keywords can be low-level and available in a testing tool out-of-the-box. For example, DoClick on an element, DoSetText into an edit field.

Keywords also can be high-level and consist of a set of lower level keywords or require programmers to implement them first.

Here is an example of a high-level keyword Login. It consists of low-level keywords DoClick and DoSetText.

And this is how it is used. User name and password are passed as input data to the keyword.

Here is the same “Login” keyword implemented in JavaScript programming language.

And this is how it is used.

In many tools keyword driven-testing stops at this point. What is beyond keywords?

Data

One of the goals of test automation is to use different sets of data in the same testing scenario. Usually data is put into external spreadsheet or text file. It is a good approach but it can be improved. What about having data and keywords/actions within the same view? It makes test development more convenient. Especially when it is easy to select what row of data to use.

In the example above the test will use second line of the data table during playback.

When you deal with test configuration it may have so many parameters that it is inconvenient to put them into a row (to avoid horizontal scrolling). It is better to put them into columns.

Conditions and Loops

If it is possible to make a test using linear sequence of steps it is better to go this way. Such tests are easier to create and maintain. Working with linear step sequences does not require dealing with conditions and loops. Typically, conditions and loops are perceived as almost programming. But they also can be created without coding. On the example below we loop through all rows of Logins table and test login to the system with different user/password pairs.

To check if login is successful or not we can use a condition.

State

Keywords and loops/conditions may not be enough to implement complex scenarios. You may need to remember some values during test execution. This is where variables are used.

Note: You may notice that we still do not write code in a text editor but the more complex our scenarios become the more programming concepts we use. This is inevitable, computers still do not understand natural language as humans do and we have to speak their formal way. If you will look at voice assistants like Alexa, Siri, Cortana or Google Assistant you will see that even these cutting-edge technologies understand limited number of sentences built in a specific way. Here is an example from things to try Alexa sends every week: Alexa, what temperature is Miami in the summer? It is a sentence built with specific pattern in mind. Testing is much more complex thing than finding information on weather conditions in Miami.

Decomposition

If scenario is long or there are many scenarios repeating the same steps then we need a way to breakdown steps into manageable pieces.

We may consider a group of steps as a high-level keyword as discussed in the Keywords section above or we can execute one test from another.

Power of Coding

If you have programming experience you still may want to use codeless approach. When appropriate there are ways of doing coding within keyword-oriented environment.

Inline Expressions

Printing number of seconds in a year.

Functions

One may create a function and program specific logic that would look cumbersome in a codeless representation.

And use it in codeless environment.

Benefits of Codeless Approach

Let’s now talk about benefits and limitations of the codeless approach.

Complexity Levels

The main benefit is that it makes possible to work at different levels of complexity.

Level 0, just record and playback
Level 1, Level 0 + modify, parameterize, make data-driven
Level 2, Level 1 + loops, conditions
Level 3, Level 2 + decomposition
Level 4, Level 3 + coding

Levels 0 and 1 are open to everyone, no special skills required. Levels 2 and 3 are accessible with minimal training or with basic programming skills. Level 4 requires programming experience.

So, any team can start doing test automation. Of course, the complexity of testing scenarios depends on experience of team members. The good news is that to operate on Level 4 a team can have just one Level 4 tester to implement all the complex stuff.

Another benefit is that codeless approach does not require to remember the syntax. There is no so-called syntax sugar and the system guides you.

Syntax Sugar

Most of programming languages including JavaScript were designed by people with mathematical background. So, this statement appears clear and simple for a programmer:

Deposit(‘John’, ‘O\’Connor’, 17.99);

A programmer easily reads this as:

Deposit $17.99 to John O’Connor

What is the difference between these notations? Symbols ‘ “ ; , . ( ) [ ] { } & $ % # @ do have meaning for language notation however are not important for understanding the matter.

Syntax sugar makes problems even for programmers. When switching from similarly looking languages some differences easily cause frustration. For example, the same construct:

$a = “Number “ + 1;

means text concatenation in JavaScript, however the same is mathematical operation in PHP.

Comparison like:

if( value == “OK” )

is good for JavaScript or C# world and produces always false results in Java.

Even if we have programming skills it is still a problem to switch from one language to another, it may lead to potential issues.

Guidance

A codeless system guides a user along the way. Want to click on an element? Choose the element from the dropdown.

Then choose DoClick keyword from another dropdown.

Want to enter text into an edit field? Choose the element, choose DoSetText keyword and enter text.

To create a condition choose If from the dropdown.

A template is generated.

Select condition and fill necessary cells.

Execute Anything

When test is being created you may need to run a specific part of it. For example, to reach the point where you’ve stopped test development yesterday, to check new steps you just added or play the test from a specific point till the end. You may need to run a high-level keyword to test it.

Custom Columns

One of the most exciting things in codeless automation is that we can extend codeless features at user level. We may define additional columns and enhance keyword-driven approach.

Every step in a codeless test has before and after triggers. A QA team may implement triggers (using programming) to give meaning to additional columns in the test grid.

The following piece of code makes snapshots of UI elements depending on values in H column.

To make a screen shot for an element before or after a specific step use sb and sa words in H column.

With custom columns and triggers it’s possible to do a lot of cool things like automatic printing of data to a report or external file or specifying IDs of requirements tested by a specific piece.

Limitations of Codeless Approach

Limitations are mostly imposed by the complexity of test scenarios. If there is complex logic and a lot of calculations are required then it may be more effective and faster to just use a programming language.

Selecting Scenarios for Codeless Automation

When you just start doing codeless test automation I recommend to start from recording and playback. Remember that not all manual testing scenarios should be or can be automated. The idea of test automation is to offload boring checks from manual testers. So, at least for the beginning, long end-to-end scenarios with switching between windows, applications and conditional logic should be avoided. When you are comfortable with short recording and playback tests proceed to data-driven testing. Then try loops and conditions, but do not overuse them. Keep scenarios simple.

If you plan on doing many tests with common steps you need an architect in the team. Someone who will design the test coverage and make a breakdown into building blocks, define high-level keywords.

From certain level of complexity you may need a real programmer in the team. Use his or her resources wisely. A programmer should implement low level helpers and make them useful for the whole team.

Domain experts should be busy with data tables and be a driving force for data-driven testing.

And remember as good accumulator should open great possibilities for e-bike owner as codeless test automation should save time for QA team and make testing easier and more effective. This is the goal.

With questions reach me on twitter @dmarkovtsev, I am always happy to help.

All examples in this article are illustrated with screenshots of Rapise test automation tool developed by Inflectra.