Making a decision: Should I Automate? Or maybe not?

“Automate EVERYTHING!” — Is this a familiar answer you get whenever you ask what needs to be included for automation testing? Before jumping in to start scripting anything that you see on the application, stop and think — does automating this test case make sense?

Well, not every test case is suitable for automation testing. Think about the ROI — you only want to automate test cases that are valuable, and you may ask “What’s considered valuable?”

In general, automation should help you reduce the time and effort for testing and thereafter allows testers to be able to focus and work on other tasks that require more manual effort, e.g. testing on newly developed features.

Scenario: Insurance Company

Imagine we are testing on an insurance application with the following business requirements:

  • There are several types of insurance (e.g. health insurance, life insurance, travel insurance, etc).
  • For each type, there are different policies/plans (e.g. under health insurance, there could be Individual health plan, critical illness plan, etc).
  • For each plan, there could be different test scenarios that are dependent on the data input (e.g. different age groups have different business flows)
  • For each form, detailed information from personal particulars to information of family members, occupation, income, etc. are required
Yes! You got the idea.

When it comes to testing, you have to fill up the same form multiple times with different sets of data in order to cover different combinations of test scenarios. After which you need to go through the same or very similar steps with different plans. Not forgetting you have to repeat everything all over again for regression testing.

Isn’t that taxing?

Automation testing helps to take over the tedious and repetitive steps, have consistent data inputs and validations, at the same time minimising human errors.

Evaluate and Plan

Before you enthusiastically automate everything, plan out where to start by finding out which insurance policy or plan is the most commonly used or has greater impact on the company should there be any failure and give those test cases the priority.

It is also important for critical business processes to be tested regularly to ensure no breakage. Having automation testing in place, the same set of test cases can be rerun multiple times effortlessly after the regression suite has been built up.

Understand the Automation Value

However, if there is any new insurance policy that needs to be delivered within a short timeframe, or if it is only valid for a short period of time before decommissioning, testing them manually may be a wiser choice. Building up the automation scripts may incur a higher effort than what you need for manual testing in such scenarios. Furthermore if whatever that is being built for automation cannot be reused or not required to be run again (for the case of decommissioning), the entire work is going to be wasted.

As for the new feature that needs to be delivered within a short timeframe, you can plan to automate the test cases at a later stage.

Which test cases to automate?

In general, here are some types of test cases that you can consider when evaluating on which test cases to automate:

  • Critical business processes
  • Most commonly used business processes
  • Test cases that are time consuming and tedious
  • Test cases that are repetitive
  • Test cases that are prone to human error
  • Cross-browser testing

Having that said, automation is not a replacement for manual testing.

So when is automation not recommended?

Aside from technical constraints or tool limitations, what other considerations do you have to take into account?

  • Short-term projects / automation scripts that are not reusable
    Features that need to be rolled out within a short timeframe or will be decommissioned after a short period of time.
  • Unstable system
    It is also not advisable to automate on features that are bound to undergo major changes or new features that are still in the process of being developed, this is especially true for UI automation testing. Automating something that changes frequently will result in a lot of overheads just to update the existing scripts instead of spending the time to build up the automation test suite.
  • Dynamic data that are not predictable
    For example, the system relies on an external system for data input and there is no way to know beforehand what kind of data is sent and the data is always different (assuming service virtualisation tool is not an option). If you don’t know what the data sent is going to be like, how will the tool know what to check?
Screen taken from Avengers: Endgame movie

Do not automate for the sake of automating!

Make your automation tests worthy. Automation testing is supposed to help you save time and effort with certain accuracy. Automate things that benefit you, your project, or your organisation.

It is NOT possible to automate everything; there’s no way to achieve 100% test automation. There are cases which require more human intervention or there may be some unknown edge cases that may break the system (which is usually covered as part of exploratory testing). If you are interested in going in depth on Manual Testing VS Automation Testing, there are plenty of articles out there. For instance, this website provided a tabulated comparison.

If you are still lost, start small. You can begin automating the most basic or essential functionalities of the system to assure that the build is stable after each deployment (e.g. navigating through all menus to ensure there is no error page).

🧙🏼‍♀ Team Merlin 💛
Application security is not any individual’s problem but a shared responsibility.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Team Merlin

Team Merlin

Software | Security | Quality enthusiasts doing the right things