ISTQB — CTFL – Chapter II: Testing Throughout the SDLC I
Fasten your seatbelts ladies and gentlemen, we’re about to take off for learning the models in software development (2.1) and levels of testing (2.2).
If we want users to like our software, we should design it to behave like a likeable person.
- Alan Cooper
2.1 Software Development Lifecycle Models
Different software development models are used to outline the order and purpose of specific activities. Each project may require a unique model and cycle, even within the same company, as it is determined by the project's needs and team. These models rely on both logical and chronological considerations.
Before moving on to listing some of the models, let’s talk about the characteristics of goods testing:
- Regardless of the SDLC model, every development activity has a corresponding test activity → as long as we are building, or changing something, we need to test and verify those modifications
- At every test level, there are some specific test objectives to achieve
- Answering what and how to test the test objective for the given test level with the corresponding development activity (e.g. unit testing, integration testing etc.)
- Involving in the review sessions (which will be later discussed in detail), refinement meetings, design and user perspective improvements
Keep in mind that starting in the early stages of the lifecycle is an important principle!
The SDLC can be divided into two categories: sequential or iterative & incremental.
Sequential Development Models
This model indicates a linear flow of the activities and it implies having sequential testing in all steps, i.e. planning → analyze → design etc. We do not start the next activity unless the current/previous one is finished. For instance, before the implementation does not finish, we do not start the testing process. This means that there is no overlap of the processes (though it’s not that practical, the sequential models suggest that).
Waterfall Model
The first software development model we’ll talk about is the Waterfall Model. The activities/steps are completed one after another and the test activities happen after all the development activities are finished. The problem with this model is that we start too late to test the developments & changes and if we encounter any problems, the steps have to be repeated.
V-Model
The second sequential model is the V-Model which gets its name from the shape V which there is coding at the bottom and as we go up, there are different test types and levels related to each step of the design and specifications. Unlike the Waterfall, V-model is integrated with the test process throughout the development process as mentioned above. For instance, at low-level design, we have unit testing, at the top we have user requirements corresponding to the acceptance testing and so on. However, the test execution does not start before the coding is finished and the customer does not see the product until the acceptance test is passed.
Incremental Development Model
This one involves having requirements, designing, building and testing in an incremental way in which we take pieces of the larger sum and break down the effort. Scrum can be an example of this but it’s also iterative. The size of each increment might change and the features like the register function can be taken as a single or multiple story. Please check this official website to learn more about Scrum.
Iterative Development
Iterative models typically have a set timeframe for completing a cycle, which may include designing, building, and testing. During each cycle, there may be additions or modifications to existing features, and each iteration contributes to the ultimate goal until a new version of the software is ready for release. It's crucial to ensure a functional product is delivered at the conclusion of each cycle. (e.g. Sprint in Scrum)
Rational Unified Process
The iterations typically last around three months, resulting in the implementation of significant features. The key is to gather feedback and adjust processes accordingly.
Scrum
This term is most likely to be encountered in the software industry as it almost took over all IT companies (and other similar ones). First, we need to get familiar with the Agile methodology — you can check out the link below which is the manifesto and twelve principles that started the Agile software development.
In the Scrum framework of Agile, each iteration is typically two weeks long and involves small developments, changes, and maintenance. The team usually consists of a Product Owner, Scrum Master, and the development team (including software testers and QA). During the sprint planning meeting, the team decides what to develop in the upcoming 2-week cycle. The development process continues alongside the testing process.
Kanban
When we look at Kanban, it can be easily seen that iteration is not the main motive but it rather focuses on incremental development. There is a board, called Kanban Board, that consists of a couple of columns (i.e. stations) like Backlog, To-Do, In-Progress, Testing, and Done. Trello is an example tool to get advantage of the Kanban model. So, it basically works without a fixed period of iteration and delivers features with single or multiple releases.
Spiral
The final development model to be discussed is the Spiral model, also known as Prototyping. With this approach, a prototype is created and then discarded for a new one to be created. The project is divided into iterations, forming a moving spiral. The Spiral model follows an incremental structure, including concept requirements, prototypes, development and test plans, implementation, and testing, among other things.
We also need to talk about the operations that occur in these software development models which are:
Continuous Integration / Continuous Delivery (CI/CD)
The thing we call CI is the process of automating the integration of code changes from different sources (meaning different developers etc.) and collecting them into a single source/project. It focuses on building and testing.
On the other hand, we also have a term called CD which deals with the deployment/release of the software which involves the automation of multiple test levels as part of the delivery pipelines. Before the deployment, running the automated (most of the time they are automated but in some projects, manual ones are also used) regression tests are crucial to detect an existing problem. It’s executed to ensure that the changes made in the code do not create any unintended breaks in other parts.
2.2 Test Levels
Test levels are groups of test activities organized together and each test level has its own process and requirements to be covered. There are four main test levels (think of them as a pyramid, starting from the bottom, we have unit testing and going up, acceptance testing occurs). It’s important to see the relationship between the test levels and development activities.
At each test level, there are five points to be discussed: test objectives, test basis, test object, typical defects & failures, and approaches & responsibilities.
Component Testing
This is the first level in the pyramid that was mentioned earlier. It’s also known as unit testing which focuses on small components that can be tested separately. It can be thought of as the smallest unit for the wider module, e.g. a function that calculates the multiplication of two integers. When we examine the five points for the component testing:
- Objective: Finding defects and reducing the risk of failures, building confidence in the product quality and verifying the requirements
- Test Basis: Detailed design, code, and data model
- Test Objects: Components, modules, classes, database modules
- Typical Defects and Failures: Incorrect functionality implementation, wrong interpretation of the logic and data flow
- Approaches & Responsibilities: Usually performed by the developer who wrote the code for that component
At this stage, it's helpful to understand Test Driven Development (TDD) and its purpose. TDD involves creating test cases before writing the unit's implementation. Naturally, the tests will fail initially. The developer then writes the code, and the tests pass. However, it doesn't end there. A final step involves refactoring the code to enhance readability and ensure adherence to standards.
Integration Testing
Integration testing involves testing how different components or systems interact with each other. There are two types of integration testing: Component Integration and System Integration. When people refer to integration testing, they usually mean component-integration testing as the second level of testing. System integration testing is conducted prior to acceptance testing and after system testing.
So, the order of the test levels can be considered as:
unit → integration (component-integration) → system → system-integration
- Objective: Finding defects in the interface between the components, reducing the failure risk and building confidence
- Test Basis: Use cases, interface, workflows, software and system design
- Test Objects: Subsystems, databases, APIs, interfaces, infrastructure
- Typical Defects and Failures: Incorrect/missing data, interface mismatch, incorrect sequencing, unhandled failures/exception
— For the ‘system-integration’ testing → inconsistent message structure between systems, failures in communication, incorrect data - Approaches & Responsibilities: Top-down, bottom-up, simulators are the approaches where component-integration testing is mostly done by developers whereas system-integration testing is done by testers
System Testing
We can determine whether to release a product based on the results of this level of testing. It examines the behaviour and abilities of the entire software or product and is commonly referred to as E2E (end-to-end) testing because it includes both functional and non-functional tests of the overall system performed by testers.
For this level of testing, it's important to create an environment that closely resembles the actual production environment where users will interact with the project. Using independent testers is recommended as they are more likely to act like actual users. It's crucial to note that this is the most significant level of testing as the reports are shared with stakeholders and the results have an impact on the decision to release the software.
- Objective: Validating the system is complete and works as expected, finding defects and preventing failures
- Test Basis: SRS, risk analysis report, epics, state diagrams, use cases
- Test Objects: Applications, HW/SW systems, operation systems
- Typical Defects and Failures: Incorrect calculations/control, failure of the system to work properly, unexpected system behaviours
- Approaches & Responsibilities: Focusing on the overall system functionality and usability, carried out by testers of the teams and independent testes
Acceptance Testing
This level of testing is the final step after system testing (although not all projects perform system-integration testing). Its main focus is on the question, "Is the app ready for the end-user?" Similar to system testing, it evaluates the behaviour and capability of the entire system, but from the user's viewpoint.
- Objective: Establishing confidence in the quality of the system, validating & verifying the system (functional and non-functional)
- Test Basis: Business process, system requirements, installation procedures, user stories, regulations and standards, business rules, disaster recovery procedures, deployment instructions, performance targets, security standards
- Test Objects: System under test, forms, reports, recovery systems
- Typical Defects and Failures: Unmatching system workflow and business rules, non-satisfactory customer feedback
- Approaches & Responsibilities: Customers, business/product owners, and testers
Acceptance tests, much like system testing, can give insight into the readiness of a product for deployment. While it's not the main goal of this level of testing, any defects found are significant as they suggest a high-priority bug that has managed to slip through all previous testing levels.
Moreover, we can also talk about 4 types of acceptance testing
- User Acceptance Testing → tests the readiness of the application/project
- Operational Acceptance Testing → tests the product from the perspective of the operational side (including security, performance etc.)
- Contract/Regulatory Acceptance Testing → checks the legal side
- Alpha & Beta Testing → There are two types of testing that are typically done in sequence. First, alpha testing is done within the company where a customer tests the product and provides feedback. Then, beta testing is conducted where the product/software is given to customers to conduct their own tests in their own environment.
In this post, we explored the various models of the software development lifecycle and their impact on the testing process. These models are closely linked to each other and affect the different levels of testing, as well as the corresponding activities involved in each level. Hope to see you in the second part of chapter two! ☺️