Test Architecture: Creating an Architecture for Automated Tests

Modern software includes a large amount of software components, and it makes the software system complex. As the system gets complicated, it is difficult to understand the product over a period of time, and it introduces a lot of risks. Software Architecture streamlines the process by arranging all components in way that they are easy to understand and execute.

Unlike development, the software testing teams might lack test architects, which will result in poor test planning, design, and strategies. Furthermore, it can increase the chances of the system failing when it is released.

The software testing industry is learning from its mistakes and slowly the testing team has started to appoint software test architects to design powerful test architecture in terms of security, performance, functionality usage, etc.

What is Test Architecture?

Test architecture is the concrete level of guidelines and recommendations to how and when to test based on the system, complexity, environment, and testing tools focused on the delivery to achieve the best possible outcome.

Test architecture might not be required for all the projects. For example, if a project is very small and doesn’t include many components, you can directly jump into the test strategy document. The small project may last only for a few months, so it makes sense not to have test architecture. However, if you are dealing with a complex project or life-critical project then the test architecture is much required.

What does Test Architecture take?

The test architecture requires in-depth knowledge of your project, domain, organization standards, governance, compliance, technology stacks that the organization is using, and delivery practices followed.

The test architect should have good expertise in testing using both automation and manual. He should clearly understand the requirements and roadmaps. The test architecture should be designed for the future so he should make valid assumptions and risks before the start. Depending on the project the test architecture may be a one-time activity or it may evolve as the project evolves, so the architecture should be flexible and robust.

History of Test Architecture

The concept of testing was introduced soon after the first software was evolved. It takes back to 1950 when Tom Kilburn wrote the first piece of software. Around 1958 the first team was formed specifically for testing. Thereafter the software testing started evolving it took various directions to streamline the process.

From 1980 to 2000 a lot of evolution happened. It took testing to a new level. During this era, the testing practices and approaches were defined, and new tools arrived specifically for testing purposes. A separate branch was started and called Software Quality Engineering. Later the test architecture, test strategy, and test plan recommendations started to evolve.

Today, Most companies have a dedicated Quality assurance team which will contain test architects, manual testers, and automation testers. Organizations have learned the importance of testing so significant investment is made in software testing resources, tools, and infrastructure.

Why create test architecture?

The test architecture is important, especially when you are dealing with a complex project or expecting the project to grow in the near future. The test architecture helps to reduce the risks and eliminate the assumptions before delivery.

As you are aware, anything you do randomly may not help in a better outcome. The test architecture streamlines the entire process of testing. Unlike other testing activities, it’s not focused on a single testing activity rather it is focused on the entire testing and the testing team aims to deliver a high-quality product. Some of the reasons why test architecture is important are listed below.

• Helps streamline the testing process
• Helps to reduce the risks associated with delivery
• It helps to define the testing ecosystem, such as the testing environment, infrastructure tools for test automation, manual testing, and anything that is part of the system.
• Helps in documenting the gaps in testing and aligning accordingly
• Brings the testing team, testing process, and tools under a single umbrella, irrespective of manual, automation, or non-functional testing.

What is the role of the test architect?

The test architect’s role is as important as development. As companies start adopting automation into their organization, the test architect role becomes more critical. Having said that, not everyone can become a test architect as it requires passion to explore and a deep level of understanding in testing and product; some roles may also demand knowledge of DevOps.

Let’s look at the test architect role at a glanceo

• One should have the passion to explore the current technologies in testing and should explore the possibilities of adopting them to the current project
• He/She should be senior enough to make the right decisions with a deep-level understanding of organizational practices and strategies.
• One should understand the product and work with the product managers and stakeholders.
• The architect will be the key person in the testing team; he should guide and mentor the team by defining the strategy, clear goals, and milestones.
• Architects should build, monitor, and adapt the new technologies throughout the product life cycle
• Identify the testing gaps and provide a timely plan to fill them
• He should also concentrate on the documentation
• Though the role doesn’t involve people management, he should ensure the team members are updated with all technical aspects and understand the process correctly

Test architecture explained with an example

The test architecture differs from project to project. The test architecture depends on a lot of factors as explained above. Let’s look at the example of test architecture at a glance.

Consider an example of any simple product. Let’s look at the general way of test architecture.

Test Pyramid

The test pyramid design is the first step while building the test architecture. The test pyramid is about different types of tests, such as unit tests, API and Integration tests, and end-to-end tests, and their percentage. Additionally, you can also add manual tests or acceptance tests into the test pyramid.

Responsibilities

Once you choose the testing types that you are going to adopt for the project, you need to decide who is responsible for what, for example, developers need to write the unit tests, and the QA team is responsible for the end-to-end tests, etc.

Tools and Infrastructure

The test architecture should clearly define the tools that they are going to use. For example for automation tests, the tool will be Testsigma; for manual test repository it will be in the Testlink, etc. Further you need to decide on the DevOps integration, execution frequency, etc.

Testing process

The testing process defines when testing is carried out on different environments such as QA, Staging, Production, etc. Additionally, you can also define when the automation scripting will be done — is it the same sprint or n-1 sprint?

The above are initial tasks, once you complete the initial tasks, the test architecture should further move on to in-depth.

The in-depth scenarios are listed below:

• Define a clear roadmap for automation, manual testing, and non-functional testing
• Design the robust automation framework
• Write the automation test cases
• Document the manual test cases
• Define the standards for automation
• Set the target for automation
• Define the non-functional testing requirement (NFR)
• Choose the tools and environment for NFR
• Define test strategy document
• Define the test plan

As mentioned earlier, there may be additional activities based on the project and product complexity. Remember that anything which is part of testing is also part of test architecture.

Test Architecture and Test Automation

The test architect role has gained importance because of automation. Automation is the key area in modern testing. The test architecture of automation depends on the framework type you choose.

• Evaluate the requirements against the automation tools, and choose the best tool for the project.
• Create the automation framework based on the industry standard principles and practices.

The framework can be keyword-driven, page object model, hybrid, modular framework, or codeless automation framework

• Document the checklist and standards that are necessary to maintain while writing the codes.
• Define the automation execution strategy.

The automation execution strategy is the broader term, it includes a lot of activities such as:

• Browser types and browser versions
• Platform or operating system and its version
• Frequency of Sanity, Smoke, and regression tests
• Suite level execution
• Executing different types of tests such as Integration, API, and End to end tests
• Integration with DevOps such as Continuous Integration(CI)/Continuous Delivery (CD) pipelines
• Test reporting and analysis

Based on the scheduled frequency, the reports will be generated.

- If you find the test case failure, how to analyze and who will do that?
- Before logging the failure as a defect, what are the necessary steps required?
- When and what to share with the stakeholders

• Documentation

Documentation is much more important but most of the team lacks. The automation test architecture document should be accurate and simple. For example, if a new joiner comes into your team, he should be able to set up and follow the process without asking anyone. The documentation reduces the knowledge transfer efforts, and it makes onboarding quicker.

• Manual vs Automation

The target for automation is set at the beginning however, it is also a known fact that 100% automation is impossible. How to deal with non-automated test cases. How manual testing is carried out etc. should be defined in the test architecture.

• Communication

The test architect works with multiple teams such as development, DevOps, testing, and business/product team. So the test architect is responsible for communication with stakeholders. If there are any challenges from the development team he should be able to work with them and get them resolved.

The complexity of the test architecture for automation depends on the tool you choose. Because some tools require creating the framework, some come with a framework ready. Not all the tools require coding, so the activities that are involved in defining the coding standards and setup will be reduced. For example, if you choose selenium you need to build the framework from scratch. You need to put a lot of effort into setting up the framework such as building the POM model, writing the base class utilities, configuring the reports, etc. On the other hand, if you choose the codeless automation framework like Testsigma everything comes as ready to use. No need to write a single line of code. Reporting will be ready, environment configuration also will be ready, no need to write the coding and checklists.

Test architecture is a broader term that includes a lot of activity. The test architecture takes care of the whole testing activity from start to end. This includes designing, implementing, documenting learning, etc.

Test architecture is required to deliver the quality product on time, especially if it is a complex project. Not anyone can become a test architect as it requires passion to learn and adopt the technologies every day. Furthermore, it is a technical role that requires in-depth technical knowledge of the product, infrastructure, testing tools, and testing process.

As the demand for automation is increasing the demand for test architects is also increasing. The complexity of the automation test architecture depends on the automation tools you choose. The codeless automation tool requires less effort to design and maintain.

Frequently Asked Questions

What is the project architecture for testing?

Test architecture is the standard term used in the testing industry. The project architecture for testing is the same as test architecture; these words can be used interchangeably.

Isn’t Test Architecture similar to Test Planning?

The test planning and test architecture are not the same. Test planning is part of the test architecture. It focuses more on the planning by focusing on a certain set of activities however, the test architecture focuses on the entire testing life cycle irrespective of the testing types.

Unlike test planning, the test architecture can be one time, or it might evolve as the project evolves. The test planning doesn’t require many technical details however, test architecture requires hands-on experience in different testing areas.

Originally published at https://testsigma.com on September 29, 2023.