ISTQB — CTFL - Chapter II: Testing Throughout the SDLC II
In this part, we will focus on Test Types (2.3) and Maintenance Testing (2.4)
2.3 Test Types
When it comes to testing software systems, there are various types of tests that are carried out to achieve specific testing objectives. These tests can be classified into four categories: Functional, Non-Functional, White-Box, and Change-Related Testing.
Functional Testing
The conducted tests evaluate if the functions of the system perform as they should. What these functions should do can be described in:
- user stories, business requirements specifications, functional requirements, use cases etc.
It's important to conduct functional testing at every test level, although the objective of each level may vary. Essentially, functional testing is all about verifying the software's behaviour. This is where the black-box test technique can be useful. It focuses on the system's input and output and not on what's happening internally (the implementation) of that function or method.
The extent to which the requirements and test cases have been covered through functional testing can be determined by measuring functional coverage. This measurement is expressed in percentage and by establishing traceability between the test cases and requirements, the coverage gap can be identified.
Non-Functional Testing
In comparison to functional testing, non-functional testing results are not always a simple Yes or No. While functional testing may yield a straightforward answer, such as whether a function correctly calculates insurance amounts based on house data inputs, non-functional testing evaluates the overall characteristics of the software system. This includes assessing usability, performance, efficiency, security, and ISO-standard applicability. Ultimately, the main question is how well the system behaves.
Don’t be mistaken about performing these tests at only specific test levels! Non-functional tests can and should be conducted at all test levels.
To determine which elements have been covered through test executions, we can measure non-functional coverage using methods similar to those used in functional testing. This includes tracing test cases, assessing design compatibility and user feedback on usability. For example, to test system performance, we can establish the highest stress level the system can handle and conduct tests to observe its behaviour under these conditions.
White-Box Testing
The important thing in the test type is the internal structure of the system/implementation, unlike black-box testing. It is more difficult than black-box testing since the architecture of the implementation & system, standard/principle applicability, data flows and more points are under the testing objective (code is not the only test object as you can imagine).
To measure how much of the internal structure is tested, we use structural coverage by expressing it as a percentage to understand how much of the structural elements are covered/tested/executed. In unit testing, code coverage can be represented as a percentage of how much code is executed and tested.
Also, for the component-integration testing level, white-box testing can be used for interfaces between components and the architecture of the system and coverage can be defined as the percentage of interfaces exercised.
Change-Related Testing
We regularly make updates and modifications to improve the quality of our features and systems. To ensure that these changes address any issues and do not cause any unforeseen problems elsewhere, testers carry out confirmation and regression testing. This helps us maintain the integrity of our processes.
- Confirmation Testing
This type of testing is conducted after a defect is fixed and the parts that failed to fulfil the requirements are re-tested (following the re-produce steps of the test cases) to confirm the changes have fixed the bugs. - Regression Testing
When solving problems in software or systems, it's possible to unintentionally affect other areas due to how packages and modules interact with each other. To catch any unforeseen side effects and ensure all components are still functioning properly, regression tests are conducted. These tests are typically gathered in test suites and automated in many projects, as there are often multiple regression tests to be run repeatedly.
2.4 Maintenance Testing
Once the software is made available to the end users in the production environment, regular maintenance activities are necessary. These activities primarily involve testing any changes made to the system (as mentioned in the change-related testing section) and scheduling the release date for any new updates or hot-fixes.
There are three triggers for maintenance testing— when the product is in the production environment:
- Modification: When something is changed/modified on the software
- Migration: When switching to a new platform or environment, including some data conversions, version upgrades and more
- Retirement: When the software/product will no longer be used and need to make sure that the database and any other related parts are safe. It also requires archiving the software
Impact Analysis for Maintenance
This analysis is closely tied to maintenance testing, which aims to assess the effects of changes made for maintenance purposes. It helps identify the intended outcomes of desired actions and can be performed prior to making a change. For example, if a customer requests a new feature for the dashboard, the answer given would depend on the results of the impact analysis.
This analysis also helps to give an estimate of how many regression tests will be required if the changes are to be made.
Now, we completed chapter two of the ISTQB-CTFL syllabus which was about the test level, types and their integration with the software development models. The important thing is to choose the correct method, technique for the required test level and project implementation methodology.
“The most expensive way to test your idea is to build production quality software.”
— Jeff Patton