Building Test Automation From Scratch

Testing is mostly about finding out how well something works. More specifically: Testing is the process of evaluating a system or its components to find bugs or identify any other issues. Testing can be done manually, or it can be automated. It is an essential part of the software development process, as it helps ensure that the software is high quality and meets the requirements of the product managers or end users.

There are many different types of testing, including unit testing, integration testing, functional testing, automation testing, and regression testing. Each type of testing has a different goal and is performed at a different stage of the development process. For example, unit testing is typically done by developers to test individual components of the software, while functional testing is done by testers to ensure that the software works as intended.

Overall, the goal of testing is to find and fix bugs and other issues before the software is released to users. This helps ensure that the software is of high quality and meets the needs of the end user.

What is automation testing?

Automation testing, also known as test automation, is the process of using software tools to automatically run and evaluate tests. This is in contrast to manual testing, where tests are performed manually by a human tester. Automation testing can be used for a wide variety of tests, including unit tests, integration tests, and regression tests.

The main advantage of automation testing is that it is much faster and more efficient than manual testing. Automated tests can be run quickly and repeatedly, allowing for thorough software testing. This can save time and resources and can help identify bugs and other issues more quickly. Another advantage of automation testing is that it can be more accurate and reliable than manual testing. Automated tests are executed the same way every time, which means that the results are more consistent and predictable. This can help reduce the number of false positives and false negatives reported. But with great power comes great responsibility.

Automation tests are one of the key elements of building successful QA processes, but there are lots of questions to be asked before starting automating tests. What tools to use, how to use, how to implement tests, and which tests to automate? These questions require a lot of research, analysis, and discussion.

We believe that, as a Recognyte QA Team, it is not considered a proper testing process unless a tester understands every aspect of the product. To be able to test a feature or part of the application, the tester needs to fully understand what that feature or part of the application does, how it works, what the services are behind it, and how is the user going to interact with it. With that knowledge, a tester can develop test cases to test it properly and also can guess possible weaknesses or where it might fail.

First Step: Creating the Test Libraries

The first step of our automation journey was the creation of test libraries. A test library is a collection of tools and resources that are designed to help with the process of creating and running tests for software. These libraries typically include a variety of different components, such as assertions, mock objects, and test runners, that can be used to validate the behavior of software and ensure that it is working as intended. The main reason behind this step was; “if there are no tests to automate, what are we going to automate?” Also, a solid base of test libraries can improve the speed of implementation of automation significantly. Because almost every test step is scripted and waiting to be automated. Another benefit is that a regression suite can be created from this library. It is my personal opinion that when a tester starts from scratch while creating the test library, it is a great way to discover and learn every aspect of the product. Because the tester is going to dig into every feature and every corner of the product to ensure that test case coverage is maximized. In addition to that, having a solid test library is quite essential for analyzing tests and deciding what is going to be automated. We’re going to touch on this point further in the next sections.

Second Step: Creating the Automation Test Cases

In the test libraries, we have almost every scenario and every test case on our hands set up now. But there is a decision to be made. What are we going to automate first? This is a very controversial subject but there are a couple of tactics to decide. When deciding which tests to automate, it is important to consider several different factors, including the complexity of the tests, the amount of time and resources required to run them, and the potential benefits of automating the tests. In general, tests that are likely to be run frequently, are time-consuming to perform manually, or are critical to the functionality of the software are good candidates for automation. It can also be helpful to prioritize the automation of tests that are likely to uncover significant issues or defects in the software. Additionally, it is useful to consult with other team members, such as developers, testers, and product owners, to gather their input and insights on which tests should be automated. Taking into consideration these factors, Our team decided to go with an end-to-end approach to our automation cases, because that approach is going to cover almost every critical feature and step in our users’ flow. Next, we created our end-to-end test cases using the scripted tests in our library. After discussing our end-to-end cases with product owners, the last touches were made.

Third Step: Selecting the Automation Tools

Using the right tool for test automation can greatly improve the efficiency and effectiveness of the testing process. The right tool can make it easier to create, maintain, and execute tests, and can provide useful features and capabilities that can help identify defects and improve the quality of the software.

However, it’s important to keep in mind that the right tool is not necessarily the most feature-rich or expensive tool available. The tool should be selected based on the project and the team’s needs and should be a good fit for the technology stack being used.

It is essential to have good tools, but it is also essential that the tools should be used in the right way. With that in our mind, it should not be a snap decision. While selecting the right tools there are a couple of factors to be considered.

• Compatibility with the technology stack: The tools should be compatible with the technology stack being used in the project, such as the programming language, frameworks, and libraries.
• Ease of use: The tools should be easy to use and learn, especially for team members who are new to test automation.
• Cost: The cost of the tools, including any subscription or licensing fees, should be considered and compared with the budget allocated for test automation.
• Support and documentation: The tools should have good support and documentation, including tutorials and user guides, to help team members get up to speed quickly.
• Integration with the development and testing environment: The tools should be able to integrate seamlessly with the development and testing environments, such as the version control system, continuous integration server, and bug tracking system.
• Features and capabilities: The tools should have the necessary features and capabilities to support the testing needs of the project, such as support for different types of tests (e.g., unit, integration, functional, performance), ability to handle complex scenarios, and the ability to generate comprehensive reports.
• Scalability: The tools should be able to handle the volume and complexity of the tests as the project grows and evolves.

After considering these factors and wide spectrum research we made our choices. But it is a well-known fact that the tools are not going to be the same forever. It’s important to continuously review and assess the tool being used to ensure it is still meeting the needs of the project. As the project and team evolve, the testing needs may change, and it may be necessary to switch to a different tool better suited to the current needs.

Fourth Step: Firing up the Automation

This might seem like the last step here but it is just the beginning of this journey. Based on my personal experience every software development process has its own unique characteristics. Therefore automating the application will be a unique process too. Lots of automation structures can be found on GitHub but they are suitable for their own applications. Just copy-pasting them will not be useful for us. Our structure could be similar to theirs but it needs to be customized and changed for our requirements. As a first step here we analyzed our application and built a structure that meets our requirements for automation. Then started writing the automation scripts. The scripting part is a bit tricky too. For example, if the application requires 2-factor authentication for the login process, as a QA you need to find a way to retrieve the 2-factor authentication code or find a way to bypass the 2-factor authentication step. In the real world, applications are not that simply coded like in the test automation tutorials or documentation.

Having automated tests on the QA Engineers’ local environments doesn’t mean a lot for the QA lifecycle. They needed to be in Continuous Integration, Continuous Deployment to deliver what is actually expected from the automation.

Continuous integration/continuous delivery (CI/CD) is a software development practice that involves continuously integrating code changes into a shared repository, building the code, and deploying it to production automatically. The goal of CI/CD is to automate and streamline the software delivery process so that code changes can be released to production more quickly and with fewer errors.

The “continuous integration” part of CI/CD refers to the practice of frequently integrating code changes into a shared repository. This helps ensure that code changes can be easily shared and reviewed by other team members, and it helps catch problems early in the development process.

The “continuous delivery” part of CI/CD refers to the practice of automatically building and deploying code changes to production as soon as they are ready. This can be done manually, or it can be automated using tools.

And automated testing is an important part of the continuous integration/continuous delivery (CI/CD) process because it helps ensure that code changes do not introduce new bugs or regressions. When tests are automated and run as part of the CI/CD process, they can catch problems early in the development cycle, before the code is deployed to production. This helps reduce the risk of deploying faulty code and can save a significant amount of time and effort in the long run.

There are several benefits to running automated tests in CI/CD:

1. Improved code quality: Automated tests can help catch bugs and other issues early in the development process, which can improve the overall quality of the code.
2. Faster feedback: Automated tests run quickly, so developers can get feedback on their code changes in a matter of minutes or even seconds. This enables them to fix issues early in the development process, rather than waiting for manual testing to be completed.
3. Reduced risk: By catching problems early in the development process, automated tests can help reduce the risk of deploying faulty code to production. This can save a significant amount of time and effort that would otherwise be spent on debugging and fixing issues in production.
4. Increased confidence: Running automated tests in CI/CD can give developers confidence that their code changes are working as intended and that they are ready to be deployed to production.

Overall, running automated tests in CI/CD is an important part of the software development process because it helps ensure that code changes do not introduce new bugs or regressions, and it can save time and effort in the long run.

Last but not least

As a final point, it is important to carefully consider the suitability of test automation for a particular project. Automated testing can bring many benefits, including increased efficiency and accuracy, but it is not a one-size-fits-all solution. Organizations should carefully evaluate the potential costs and benefits of test automation and determine whether it is a suitable approach for their needs.

In addition, it is important to remember that automated testing should be used in conjunction with manual testing, rather than as a replacement for it. While automated testing can be useful for performing repetitive tasks or testing at scale, it is not able to replicate the intuition and experience of human testers. By combining automated and manual testing, organizations can ensure that their systems are thoroughly tested and that all potential issues are identified.

Finally, it is crucial to establish clear processes and protocols for maintaining and updating the automated testing system to ensure that it remains accurate and effective over time. This can involve regular updates to the test scripts and maintenance of the automation tools, as well as ongoing training for team members who will be using the system. By investing in the ongoing maintenance and development of the automated testing system, organizations can ensure that they get the most value from their investment in test automation.

Semih has 6 years of experience with full system development life cycle experience including designing, developing and implementing multiple test cases and plans in software and web applications, working in various fields including cyber security, webRTC and real estate. He currently holds the Senior QA Engineer position at Recognyte.