Guideline to improve test cases on Testing Strategies based on Maintainable Automated Test Suites

Introduction

The goal of this guideline is to improve the time that needs to be invested from the Automation team to understand and implement the functional test cases.

This is particularly important for Testing Strategy where automation process is performed one sprint later than functional testing and/or Regression , or if E2E test cases are automated in a later stage. This will allow the team to go through this process without wasting time and with an effective result.

Here you can find some tips on How to convert functional test cases, that were based on functional knowledge, into a test case that can be implemented by any person that needs to automated it.

The first thing that you need to have in mind is, Why do you as a functional tester want your test cases to be automated in the near future?

Here you have the main reasons you can have:

• I’m bored of doing this every release.
• I don’t have enough time to do everything I have to do.
• This is something that takes me a lot of time.
• I want to reduce my participation on this specific testing project.

And the answer is as easy as: if there is a good automated test base that you can rely on, then you can move forward.

How can I trust an automated test case as a functional tester?

The best way is to be involved in every phase of the automated test creation. From the definition of the test itself, scenarios that need to be covered, the acceptance criteria of the test and finally performing a functional review of the script after it is finished.

How can I improve the quality of the test?

You need to keep in mind that in order to consider a Test Case to be ready for automation, it should usually contain more information than the one usually included on a test case that has been written to run it during a manual regression.

A good way to understand the information that is needed to add inside the test case is to follow the Done Criteria or Acceptance Criteria that should be applied for the specific automation project. Different projects should require different Done Criteria and also different information.

Example: about the relationship between a done criteria ( must column ) and How to improve the test case. Remember that each project should agree with the Done Criteria related with the specific requirements inherent in the project.

Here some tips directly related with the criteria set above

Maintainable

There are a lot of examples of automated test cases that could be run just once and the next time you need to re-check it, you need to create a new set of data in order to be able to execute it again. If the data is hardcoded, it is usually faster to execute them manually than updating the scripts. As a consequence the scripts are not going to be used anymore by the functional team.

In order to avoid that, as part of the preconditions of a test case is a good practice to provide a way (per example a query) in order to supply the necessary test data.

Example: If the test case needs to run with a specific role. Do not set only the role name but also the way to find that information. It could be a database query, or a list of users if the environment is not changing frequently. Always thinking on minimizing maintenance and also in this way it is not needed to have business knowledge in order to develop the script.

Incremental Coverage

Adding parameters to the scripts could be a good way to improve the coverage of the test cases, without duplicating unnecessary code.

Example: The flow is related to checking if a user is active or inactive. Suppose that usually the test runs by randomly picking users to check.

If there is information related to the age of the users or the gender, that could be used as a parameter that can be fulfilled in order perform a testing specifically focused on that section of data in that specific test suite run.

Parameters are very important in the data creation performed by script .

Extensible

Adding a list of the scenarios that affect the flow is very important, in order to cover not only the happy path of a functionality .

Remember also that if a scenario that was not covered during the testing process is detected in UAT or production, it needs to be added to the test plan in order to catch it in advance the next time .

Coverage of the functionalities

As it was mentioned on the above table, one of the tips to improve the win-win relationship between functional and automated testing is the independence of the teams.

As a good practice, include the screen map on the summary of the test case. This is the route that the user needs to perform to arrive at the starting point of the specific workflow. This will ensure that the script is being developed inside the correct business flow.

This is very important in complex projects where you have multiple entry points for the same functionality. For instance, if the checkout can be performed from the details page and also from the order page. To avoid any confusion, add the screen map.

How to create this?

Add a breadcrumb: put in a simple sequence the pages/steps needed to be performed in order to arrive at the start point.

Example:

Login->Home Page-> Classroom Tab->Student Subtab

This means that the user needs to Login and at the Home Page , select the Classroom Tab. Inside the classroom tab go to the Student Subtab.

Steps

Steps should be written considering that the person who will automate it may not have the needed application’s/business knowledge. Expected Results and validations should be added step by step.

Example: if there is a drop down included in the flow, it is needed to add a query to validate the options that should be displayed on it, or the list of values . Bear in mind that if you add the list of values and that list change in the future, the test case will require maintenance. If you add the query to find the options, it will not be needed.

How to Identify a test case for automation?

It is important to understand when a test case needs to be automated because of the level of effort that requires to perform that task.

1. Based on Parameters

Remember that parameters depend on the specific project/application you are working on.

Here some examples:

• Test case executed with different sets of data
• Test case executed on a different browser
• Test case executed in a different environment
• Test case executed on different set of users
• Test case that Involves a large amount of data
• Test case that has dependencies
• Test case that requires Special data.
• Test case than needs to be included in every regression
• Data creation.

If your project has any of these scenarios is a good option to automate those test cases involved.

2. ROI (return of investment)

Commonly used as a tool to discard some test cases in order to automate

Basically, it is a measure of the time savings of having automated scripts running instead of manually testing.

The ROI should be high and incremental to consider the automation of that test case as a good investment.

Example:

A test case took to run manually 5 minutes and it was assigned to 2 regression plans during the last year . (Assuming the team has 1 release per month).

Due to the complexity of the preconditions of the test case, automation of it would take 24 h.

Is it considered a good candidate for automation?

Answer : NO

As a guide and agreement between the functional and automation team you can set the Functional Review ‘s Acceptance Criteria.

Remember that the Acceptance Criteria always depends on the project, sometimes you face some limitations regarding the data available, the infrastructure, time or budget. Have into consideration all these inputs before asking for goals that are impossible to achieve.

Here a very graphic example about Acceptance Criteria