Maintainable Automated Test Design

Design automated test scripts that are easy to maintain

Zhimin Zhan
Feb 26 · 6 min read

All test automation engineers know that a single application change may break many automated test scripts, and these kinds of application changes happen constantly. To effectively deal with it, test automation engineers need to be in a position to respond to a simple application change in a matter of seconds (here I mean the time is taken to modify the test scripts, excluding execution).

The key to test maintenance (a common problem in test automation) is the test script, not the test tool. Tool vendors like to focus on their tools or deliberately blur the relationship between test scripts and testing tools, for the sake of commercial interests. It is the test engineer’s knowledge that ultimately determines the quality of the test scripts. Good testing tools should help tester engineers transform their knowledge into reality, efficiently. But first, the test engineers need to have that knowledge.

“Expensive Tools Do Not Produce Better Designs” — The Pragmatic Programmer book, Tip 59

To be able to maintain a large number of automated test scripts that can keep up with application changes, we need to design our test scripts that are intuitive and can be updated efficiently by the testers (or even business analysts).

  • Intuitive to read

Generally speaking, testers are not tech-savvy. Therefore test scripts must be intuitive, and ideally readable. How intuitive should (or can) test scripts be? If we step outside the software development, consider how people talk about web sites, you might hear instructions like this:

  1. After login
  2. you shall see an account summary page
  3. on that page, click ‘select pay by credit card’
  4. Enter your credit card details on the payment page
  5. Record the receipt number on the confirmation page

These steps set a good target for our test script syntax. More on this later.

  • Easy to update

The core reason for hard-to-maintain test scripts is duplication. It is very similar to the problems caused when software code is duplicated (often by copy-and-paste). It happens too often when changes are applied and people forget to update all the references. In the context of test scripts, duplications are often widespread in the recorded test steps. When applications change frequently (almost invariably), duplication will lead to maintenance difficulties.

DRY stands for Don’t Repeat Yourself, that is, Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.
— “The Pragmatic Programmer” by Dave Thomas and Andy Hunt

You might have heard the term “DRY”. DRY in test automation means if one test step’s corresponding operation changed in a web application (such as the button ‘Confirm’ on the payment page being renamed to ‘Pay now’), only one place (in test scripts) needs updating, regardless of this test step being used by many test cases.

Maintainable automated test design

Let’s revisit the sample instruction of how people talk about the use of web sites:

You might have noticed I highlighted the login and several XXX pages, which are examples of two key concepts for maintainable automated test design: reusable function and page object.

1. Reusable function

As we know, a user login function commonly consists of three steps:

  1. Enter a user name in a text box
  2. Enter a password in the password box
  3. Click the ‘Login’ button

For this frequently used function, we prefer not to repeat these 3 test steps every single time (which would be a good example of causing duplication) in our test scripts.

Here is a sample test script (in Ruby, the design works for all languages, see the link below for examples in other languages) using a reusable function for user login.

The function’s definition:

These reusable functions are defined in a test helper ( I often name it ‘test_helper.rb’ ), which are shared by all the test scripts, i.e. reusable.

2. Page Object Model (also known as Page Pattern)

“If you have WebDriver APIs in your test methods, You’re Doing It Wrong”
— Simon Stewart, lead of Selenium WebDriver project (link)

Page Object is a common test design pattern (in fact, list as a best practice on Selenium documentation) that groups a web page’s operations into a ‘logic page object’ (feel confused? Stay with me, it will become clear after you see some examples). Here is a sample test script using a page object credit_card_page:

The page name credit_card_page reveals the purpose of operations on the page, enter_holder_name and enter_card_name are two operations on this page.

Page Object can be a difficult concept to understand initially for people who haven’t programmed before. It is important not to get them to learn Object-Oriented concepts and programming, which are not necessary. Instead, guide them with an easy-to-understand exercise, like the one in the “Practical Web Test Automationbook. From my experience of training and working with numerous manual testers, with a carefully designed exercise and a good testing tool, everyone can master it quickly (in minutes).

Maintain with ease

Besides syntax benefits (easy to read), test scripts that make use of reusable functions and Page Objects are much easier to maintain. This is because test steps are now in reusable functions and the operations are in page objects. Test scripts in (top-level) test cases are not referencing test steps directly. Instead, it becomes essentially a two-tiered arrangement.

When a change is introduced on a web page, its corresponding test steps need to be updated accordingly. If the affected test steps are used directly in many test cases, we need to update them one by one, which is tedious and error-prone. If test cases use page objects, there is only one place that needs to be updated — the function in the page class.

For example, after identifying the credit card page was changed in the new build, we just simply navigate to the definition of page class: CreditCardPaymentPage and update the test steps for affected operations there. Depending on the nature of application changes, test cases themselves might not need to be updated at all.

Sample test script: https://github.com/testwisely/agiletravel-ui-tests in several languages, such as RSpec (Ruby), Cucumber (Ruby), Mocha (JavaScript), and PyTest (Python).

Sample test project (opened in TestWise, can be in others as well) :

I have been using the same test project structure (very simple, right?) for over 10 years. The test suite for my own app WhenWise has about 500 Selenium tests, the same project structure: one test helper, one pages folder containing many page classes (pay attention to the scroll indicator in the left project pane).

Can you identify the reusable functions and page classes in the above test script (displayed section)?

  • reusable functions
    — business functions:sign_in
    — general web-testing functions: visit , try_for (for testing AJAX)
  • page classes: FindBusinessPage , BookResourceCalendarPage , BusinessSearchPage, ConfirmBookingModalPage

Work with Maintainable Test Design efficiently

To efficiently develop/refine your test scripts in maintainable test design, use Functional Test Refactoring.

(This article is an excerpt of my book “Practical Web Test Automation”)

Geek Culture

Proud to geek out.

Sign up for Geek Culture Hits

By Geek Culture

Subscribe to receive top 10 most read stories of Geek Culture — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Zhimin Zhan

Written by

Award-winning software developer, test automation & CT coach, author, and speaker. Help teams succeed with Agile/DevOps by implementing real Continuous Testing.

Geek Culture

A new tech publication by Start it up (https://medium.com/swlh).

Zhimin Zhan

Written by

Award-winning software developer, test automation & CT coach, author, and speaker. Help teams succeed with Agile/DevOps by implementing real Continuous Testing.

Geek Culture

A new tech publication by Start it up (https://medium.com/swlh).

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store