ISTQB — CTFL - Chapter I: Fundamentals of Testing I
As mentioned in the last post, the first chapter of the ISTQB — CTFL Syllabus (Version 2018 v3.1.1 and Version 4.0) is the fundamentals of testing and I’ll explain the topics in two parts. This post will focus on the first two sections 1.1 What is Testing? and 1.2 Why is Testing Necessary?
1.1 What is Testing?
The software has become an integral part of our lives. Before approving a particular software or application, we go through a testing process to ensure that it functions as intended. In essence, we aim to evaluate the quality of the software based on predefined standards and requirements and minimize the risk of any operational failures in the product or service.
As a software tester, your goal is not to improve the quality of the software but rather to ensure that it meets the intended standards. It's crucial to determine what those standards are.
Software development involves multiple steps and procedures. Testing can be done by running the software (dynamic testing) or reviewing certain parts (static testing). This ensures that the produced software is verified (rechecking things that users may not consider important) and valid (checking specific things that users have specified).
The Objectives of Testing
- Preventing defects through evaluating the work products, reviewing the user stories/epics, reading the user requirements specification document etc.
- Providing information to the stakeholders, i.e. writing a process/status report
- Building confidence in the test object (it can be the overall software like a mobile app or just a part of that app like the function that calculates the prices of the items in the basket of an e-commerce app regarding the discounts etc.)
- Making sure that we followed the requirements collected from users, stakeholders, technical leads, business providers etc.
- Following the laws, regulations such as a contract
- Findings defects and reporting them appropriately
Note that, the objectives can change depending on which projects we are working on, who we are working with, what’s the context and time etc.
Testing and Debugging
It is crucial to distinguish between two terms: testing and debugging. When we perform tests on software by following a specific path, we often encounter bugs and failures caused by defects. The developers then debug the problematic section of the code to identify the source of the issue and solve it. The tester then conducts confirmation testing to verify that the issue is resolved. If the issue persists, the process is repeated.
It is uncommon for testers in Agile companies to participate in debugging and component (unit) testing, although there are some instances where they may be involved in these tasks.
1.2 Why is Testing Necessary?
Simply state, reasons are directly linked to the objectives of testing. Testing reduces the risk of failures, finding defects and fixing them increases the quantity and requirements that must the met during the software development test cycle (SDLC).
When discussing the importance of testing, it's important to note how it contributes to the overall success of software. Test engineers and quality assurance professionals can participate in requirement review meetings to identify any additional user needs and utilize effective testing techniques to minimize the risk of incorrect implementation.
Testers benefit greatly from collaborating with designers, as it enables them to gain a better understanding of users' perspectives and expectations regarding the development process. This, in turn, is crucial for effective resource management, especially when done at an early stage.
As part of the testing process, the tester may employ static testing to analyze the code and determine the optimal testing approach, although this may not be feasible for all companies. By conducting validation and verification, testing significantly enhances the chances of the software meeting stakeholders' requirements and mitigates potential failures prior to product launch.
Quality Assurance or Quality Control?
Both of them fall under quality management which is related to meeting the criteria mentioned by customers, business providers etc. and all the relevant activities that are directly or non-directly related to this.
The goal of quality assurance is to align with the correct process flow. By following all the steps outlined in testing procedures, the likelihood of achieving high quality is greatly increased. The process begins with gathering requirements, reviewing details, designing, and testing the developer's production and iteration processes while incorporating feedback.
On the other hand, quality control focuses on the product itself as a part of the quality assurance process. It involves test activities which help to maintain some level of quality.
Errors, Defects, and Failures
The terms error, mistake, defect, fault, bug, and failure are used interchangeably these days but they have slightly different meanings and places/contexts to use each of them.
When the developer writes the code, understanding the requirement correctly (if it’s been specified properly — gathering from the customer is also significant) is crucial because implementing the wrong logic is the first mistake or error in this chain of catastrophe. After this, the testers ought to find defects/bugs since the logic, design or business rules are wrong. But let’s say the tester missed that bug which might lead to an error or failure in the software. I’m saying might because it requires certain conditions (environment, data, user behaviour etc.) to observe that failure in the production.
The reasons behind this chaos can be time pressure, miscommunication, lack of experience, gathering wrong information, environmental conditions, not being familiar with the new technology, high-complex products and processes and so on. Being a human is a reason in itself.
Identifying and Labelling Defects
When it comes to identifying defects, testers sometimes fail to label it correctly and the failures can be either a false positive or a false negative.
→ Let’s start with setting some ground terminologies straight. ‘Positive’ means that we found something and ‘Negative’ means that nothing wrong is found in the test object.
→ False Positive: Finding a defect that’s not actually in the software which can happen due to bad network connection, wrong interpretation of the requirements etc. We think it’s a bug but it’s indeed OK
→ False Negative: Not finding/missing a defect that’s in the software which can happen due to lack of focus or experience and so on.
Root Cause Analysis and Effects
When we found a defect, we investigate the reason, the deepest reason, behind it and call it the root cause. The process of doing it is called the root cause analysis which helps us to take early actions and prevent failures.
It can be considered an improvement process and it’s applicable to most of the topics (does not have to be software-specific but is mostly used in the software industry).
Can you think of a scenario where a business analyst (BA) wrote an inaccurate method for calculating insurance amounts for a real estate agent? If so, imagine that this flawed data was then used by a Product Owner (PO) in an Agile company to create a story (for example: "As a Real Estate company, I want to accurately calculate the cost of purchasing a house"). The developer responsible for the task then implemented the story.
The real estate company's software made an incorrect calculation that resulted in them losing millions of dollars. The failure was caused by a BA taking incorrect notes during a customer meeting (which is the root-cause analysis).
The point is to find the problematic part, whether it’s communication, technique, procedure etc., and take action to meet and/or improve the quality of the software and the process.
“Testers don’t like to break things; they like to dispel the illusion that things work.” — Kaner, Bach, Pettichord