A New Approach for Software Test Automation
The essential building block of manufacturing high-quality software is to conduct software testing accurately and effectively. In today’s world which we constantly try to automate everything, manual testing increases the time spent for testing and more importantly the risk of missing errors. We try to automate manual testing as much as possible. However, we come across with the difficulties of test automation itself.
Actually, what we do with test automation is writing new scripts again and it is like trying to catch the tail. It is because we worry about the correctness of the automation codes, too.
Although we can ignore the catching the tail situation, we still have to meet the following challenges:
- If you do not know how to code, you cannot find yourself a place in test automation.
- Test automation code can differ for each test. Or more than that, it should be re-coded! Maintenance can be more difficult than coding.
- If we did not manage the data to be used in the test while coding, it means that we will have hard times ahead.
- Other challenges include preparing the test environments, distributed application, well recording the outcomes etc.
We have strived hard on this field like many other experts and just decided to focus on a method which we hope to face fewer problems with. As a result of these studies, we have created our test tool, TEDAM (Test Data and Automation Manager).
Let me explain briefly to those not having much time. What TEDAM provides is to prepare end-to-end test automation for web applications, to run and manage these automation scenarios. TEDAM does not need coding to perform all these tasks. Users select the actions, relate these to their data and run on the interface we provide. In case of any updates, the user can continue using the automation by updating the test step and data. Through REST API, users can integrate their tests to CI/CD tools.
The main function of TEDAM is to automate the functional tests of the SUT (System Under Test). Thanks to the dynamic pattern of TEDAM, you can also run load tests.
We can now provide comprehensive information about TEDAM to those having enough time.
Why Test Automation?
If you have already followed us here, you apparently needed test automation. We can list the general benefits of test automation below:
- Ensures that regression tests can be run more often than manual tests.
- Compatible with agile software development methodologies.
- Supports the preparing test scenarios.
- Able to catch the details which the user may miss.
- Increases the depth and scope of the tests.
- Enhances the trust for the application’s working.
Problems and Difficulties of Test Automation
Software test automation has its own life cycle. Within this cycle, it won’t be wrong to say that the tester as the user of the cycle and the toolset used in testing are key factors determining the efficiency of the work.
General test automation needs roughly the following life cycle;
- Writing test scenarios,
- Coding the automation scenarios,
- Preparing the test environment and data,
- Running the automation scenarios,
- Assessing the outcomes and maintaining.
Anyone performing software tests can write test scenarios. What is more, scenarios can be written by an analyst, a product specialist or even a support specialist. However, is it normal to expect these people to code? What happens if your customer requests to define and automate their own test scenarios and run them in version updates?
Record and play tools? Yes, there are many tools records your actions and playing them at any time you would like to. These tools basically transform your actions into codes. What enables the code to work when it is replayed is determined by the coding competency of the tester managing the test. Test automation structure which is well built on a strong methodology can be sustainable with the testers who have coding knowledge.
Embedding the data in the test automation codes, in the scenario aggravates the controlling of the data after a certain point. Some methods such as data-driven testing have already been used to solve this problem. The aim is to differentiate the data from the code. A well-differentiated data and code mean more maintainable automation.
We have prepared test automation codes and data correctly. Now, it is time to run these. Test automation scenario set composed of a few tests can be run in a short time. What if we have hundreds and even thousands of test automation scenarios? Will it take days and weeks to run on a single computer?
In a well-structured test automation process, we need a mechanism to run our test automation scenarios in short time periods. We need to run the tests distributively and log the information such as on which version and with which data the test is run.
Introduction to TEDAM
TEDAM, an abbreviation for “Test and Data Automation Manager”, has been developed to solve the problems we face in automation and is still in progress.
What we have solved while developing TEDAM are as follows;
- Creating a structure to differentiate the data from the code,
- Centralizing the environments on which the scenarios are written, the automation codes are prepared, the automation scenarios are run, and following these environments up through a single tool,
- Ensuring those who are incapable of coding (test specialists, analysts, support specialists, product specialists and even customers) can do automation, too.
Why prefer TEDAM?
You come across a hundred of products when you search for a test automation tool. There are many categories such as free, open-source, expensive, very expensive, web, mobile, desktop, API etc.
Let’s list the features TEDAM offers.
Free. Open Source. TEDAM is a free and open source product.
Codeless. You can perform codeless automation as it is highly trendy nowadays.
E2E (End to end). You can track your test scenarios on TEDAM and even manage the clients on the tests will be run. You will have the chance to manage all your needs over a single product.
CLI Support (Command-Line Interface). With TEDAM, you can run any of your commands at any time and even by before/after automation triggers. For example; building and running Docker containers, restoring the database backup etc. Yes, you can do it all.
Notification. You can generate test results reports in excel format and by e-mail. Besides, you can add any platform you like (Twitter, Slack, WhatsApp etc.) with Notification interface.
Customizable. You can define any components to be tested in the product by your own rules. You can define all HTML components which are containing ‘textField’ in ID attribute by one rule only. Or more generally you can define all input elements by coding a single rule. Thanks to its flexibility, you can easily perform automation for HTML codes involving the components even with the same ID.
Independent. With TEDAM, you can automate your web product testing. What about your desktop applications? All you need is to select framework such as Winium and make necessary developments. Or you can integrate into mobile automation through Appium.
How to use TEDAM?
Let me explain how TEDAM works.
The process in TEDAM starts when the user collects the snapshot files from the SUT. And what is a snapshot?
Snapshot is an XML file which is in the HTML page to be tested and involves the components the user can interact and the data belonging to these components.
Sample content of a snapshot:
<Control tag='city' title='City Name' type='textfield' value='İstanbul'> <Control tag='phone' title='Phone' type='textfield' value='90245663223'> <Control tag='save' title='Save Form' type='button'>
You need to install first the TEDAM Snapshot Generator which is a Chrome Extension to generate snapshots. Characteristic features of the components to get interacted are defined on TEDAM Snapshot Generator by rule definition. When conditions are ready to start; differentiating related to the defined rules is performed on the HTML page by creating a snapshot on the given page and the suitable components are transferred to the snapshot.
We should record each snapshot file for any interaction we have had in this way for the scenario to be tested.
It is time to write our test scenario. We record our test cases through our data entry interface we call as TEDAM Face. The critical step here is to separate the test steps atomically. On the page either we enter data, or we just click. We should record all these as independent steps each.
After specifying the scenario steps, we need to relate our saved snapshot files to the related test step. When this relation is established, we will choose the action type according to the operation, the component and data to interact.
For example; if we are to click on a button on the page, we will choose “ButtonClick” step type and select the requested one among the buttons listed on the pop-up window. Or if we are to fill-in some inputs on the page, we will choose “FormFill” step type and select the fields and data to fill on the pop-up window.
The test scenario is finally ready. Next step is to categorize the test scenario under Test Sets by the correct groups.
Test Set is used to manage the relations if there is a relation between scenarios, for example; if there are precondition and postcondition scenarios belonging to a scenario.
Jobs record the test sets to run, the test environment of the product to be tested, the date when the run will start and TEDAM Agent info on which the test will run.
As everything is ready, we will trigger the Jobs to work. To be precise, we will forward these Jobs to TEDAM Manager and make them get in the queue. We can do this in three ways as follows:
· When Jobs started by a user via TEDAM Face interface,
· When it is time for planned Jobs to work,
· When Jobs triggered from REST API,
TEDAM Manager starts the work by distributing the Test Sets in the Job of which turn has come in the queue to the suitable TEDAM Agents.
When automation run is completed, TEDAM Manager notifies the user by its own notification interface.
Technology Stack
Java, Vaadin 8, Hibernate, SpringBoot, WebSockets, Chrome Extension, BeanShell Scripts and of course Selenium.
One can ask why BeanShell is included here. Let me explain immediately. Our target to automate a java applet product which was not developed on the web, through TEDAM could be realized through BeanShell within the company as we are not supported by Selenium. Actually, products not developed on the web can be automated by making the necessary developments on TEDAM. For instance; you can automate desktop products tests to test desktop software with Winium scripts.
As a conclusion…
We have reached our goals substantially. In our company, thousands of test scenarios are run on 3 different products automatically by using TEDAM.
TEDAM has still developable aspects like many other products. We are actively developing it. We have just located it on GitHub as an open source so that anyone can use and even develop it.
TEDAM can be one of the best tools in this field if its self-enhancement capabilities are better developed by adding a bit of artificial intelligence.
Hope to see you here for further development and contribution.
https://github.com/logobs/tedam
Türkçe için, buraya tıkla!
