Planning on Adopting Test Automation?
Introduction
Most software applications today are written as web-based applications accessible via internet browsers. The efforts and effectiveness in testing these applications vary among companies and organizations. Manual testing would require more time and resources, sometimes both. It is less accurate and can be cumbersome when it comes to repeating the same test with multiple datasets. For each and every test, executing manually would require human interaction with the Application Under Test (AUT) and, not to mention, can be downright dull and boring at times!
Test automation can assist with eliminating human errors and cut down time spent on repeating the same tests with different types of datasets. Automation also plays a key role for companies who are adapting to some aspect of Agile methodology. Using an automated testing tool potentially reduces manual efforts in performing repeatable user actions on an AUT. This would include (but is not limited to) Unit, Functional, Integration and Database testing.
Advantages of Test Automation
There are many advantages to test automation. For the most part, test automation advantages are related to the repeatability and speed at which the tests can be executed.
Test automation supports the following key areas:
1. Regression testing
Writing effective regression automation test facilitates validating that areas of the AUT continue to function as expected, even after some changes have been made to the system. It is a way of validating existing software without having to go through human interactions, thus, cutting time and efforts.
2. Rapid feedback to developers and/or stakeholders (product owner/product manager)
In essence, regression tests provide a rapid response should new changes break previously stable sections of the application. Using manual testing approaches require a longer time and possibly miss opportunities to test all areas of the application where new integrations may have had an impact.
3. Virtually unlimited iterations of test case execution
This is pretty much self-explanatory. Automation can help with creating any N number of test case containing N number of input, output, and validation with the expected output.
4. Support for Agile development
According to an article on Automated Testing of Agile Projects vs. Waterfall Projects, “automation is a critical component of maintaining agility and is a priority for the entire team to establish practices/disciplines and focus on continuous improvement. Continuous Integration (CI), unit, functional & integration test execution and continuous/automated deployment are common examples of applying automation beyond the scope of traditional automated tests.”
Therefore, anyone who does any action repetitively more than 2–3 times in a short period of time should automate those steps to improve the process, preferably with a well-known tool, like Selenium. Tests generally shoulder the development of functional test automation since that activity parallels the actual features/business requirements that are being developed, keeping in mind, that one-time tasks and exploratory testing/edge cases should not be automated.
5. Disciplined documentation of test cases
While scripting functional test may be one part of test automation, however, before implementing any unit test for black-box/white-box testing, understanding the business need is very imperative. Creating simple documents which explain the “need” for new functionality would provide a context of what the implementation is going to accomplish and can be used as a reference for the future. Is this an edge-case scenario or an implementation for only a limited time? Evaluating the need for automation based on these two questions, perhaps more, will provide a road map for developing use case scenario and a scope for testing. In addition, you’ll gain a better understanding of how the software functionality works with the expected results and of the actual outcome and be able to validate system behaviors in accordance with requirements.
6. Customized defect reporting
Even the most thorough test scripts may prove insufficient without helpful reporting to provide test results. Without reporting, all the efforts put into writing and executing test cases will be lost when there is no way to track and communicate information to others. Depending on the automation tools or open source libraries, there may or may not be an integrated reporting tool.
Customized reports can be as simple as writing expected input/output results into a simple text format, or as complex as writing to a DB and reading those results to HTML pages. However, there are open source libraries like the Extent Reporting library where pass/fail test criteria can be written so that technical and none technical users can figure out why a specific test case failed. Some such possible reasons for failure include “dead elements” on the page or an “expected criteria” did not match.
7. Finding defects missed by manual testing
More often than not, when executing manual test cases, testers may forget to cover all the test scenarios which may include test data, role-based, or systematic testing. Manually implementing tests with a combination of multiple variables may be cumbersome, and may or may not validate a positive outcome for all test scenarios. This is why implementing effective automation scripts can alleviate the systematic approach to capturing all the test case scenarios and possible failures of the system.
8. Removing human error on systematic testing of large data sets
Building from the previous point, it is not practical to use a systematic approach to test large sets of data manually. A systematic approach to manual testing is prone to setbacks like: failure to meet deadlines for each and every scenario and outcome; or human errors with data input vs expected output. Finding an automation solution for all the different types of input variables not only provides systematic testing, but it will also reduce human time, effort, and errors while manually testing.
To Automate or Not To Automate, That is the Question
Is automation always advantageous?
A simple answer is no. There are times when manual testing may be more appropriate. One assumption is that the User Interface (UI) will change considerably in the near future. When UI changes happen then any automation created for the impacted area would need to be rewritten. In addition, it’s not appropriate to create automated test scripts for edge-case scenarios, such as where the scenario is limited to time, impacts only one client, or contains any custom content. Another is if the project sprint has a very tight deadline between designing, developing, reviewing and testing, then it’s imperative that testing gets done within that time frame and manual testing would be the best solution. If developing automated test cases takes the same amount of time coding and debugging as manual, then to avoid time crunch, continue with manual execution of test cases and come back at a later time for implementing automation for regression testing.
Enough Talk… Give Me Automation Solution
Unfortunately, we don’t yet have a single all purpose and all organization tool that will handle every given scenario. Bummer!!
In My Humble Opinion (IMHO), when deciding on which tool to use for automation you should consider two important things:
1. How much support is available from the community of users and developers?
2. What tools and technology are in use by other companies that solve the same needs of developers and automation engineers at your company?
So, what you are saying is we are SOL?
No…, not in the least bit!
Choose a testing tool(s) that provides the best solution and ROI for the company and/or organization as a whole. Is your company willing to provision and pay per license, or are they looking for alternative solutions? How much development knowledge does the tester/developer possess and how much of the knowledge gaps can be bridged from manual-to-automation between testers and developers? How much flexibility does the tool have to accommodate customizations?
My Recommendation
Selenium is one of the most prepared solutions among the various Test Automation Tool because:
1. It’s Open Source
2. No license fee
3. Supports multiple programming languages such as C#, Java, Ruby, and Python to name a few
Most importantly, within your organization, you would find support from the developers, as well as, external forums where you would be able to find support for programming languages that Selenium supports.
