Software Testing And a Brief Explanation Of The 7 Testing Principles
Until this day, there are so many definitions of testing.
- The process consists of all lifecycle activities, both static and dynamic, concerned with planning, preparation, and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose, and to detect defects. [ISTQB]
- Testing is questioning a product to evaluate it. [James Bach]
- Testing is an empirical technical investigation done to provide stakeholders, with information about the quality of a product or a service. [Cem Kaner]
- Testing is gathering information by making observations and comparing them to expectations. [Dale Emery & Elisabeth Hendrickson]
- Testing is a way to identify defects in an s/w product.
- Testing is a process of Verification and Validation.
- Testing is defining a proper shape for a project or product from its irregular shape.
In its most basic form, we can say that it is the process of examining whether the software meets the expected features. Testing starts with the idea of implementing the software and continues with the delivery of the software to the end user after it is delivered. Our aim with software tests is to ensure that the product is error-free at the maximum level and to reveal the best quality product. In other words, testing is a work in progress throughout the entire life cycle of the software.
Let’s take a look at the 7 principles.
1. Testing shows the presence of defects, not their absence
Testing reveals bugs in the software. The purpose of performing the test is to reveal errors, not to prove that there are no errors in the software. The software is tested from many aspects by applying various methods and strategies. As a result of these tests, the main purpose is not to prove that the software is error-free. It is to find as many errors as possible. Even after many tests, we cannot say that the software is completely error-free.
2. Exhaustive testing is impossible
It is impossible to test the software 100%. It is not possible for us to test all inputs and results. Our aim should be to apply techniques based on risk assessment, where we can get more feedback from testing. It is important to focus on the potentially risky aspects of the software under limited time and resources and to ensure that scenarios cover as many points as possible.
As long as we continue to test all possible scenarios, the software execution time and cost will increase. To avoid extensive testing, we should consider some key test criteria impacts, such as risks and priorities, as part of our testing and forecasting.
3. Early testing saves time and money
Early detection of errors results in significant cost reductions. Early detection of any flaws in requirements or design ensures both faster correction of the error and no repetition of the same error.
At Trendyol, we know that early testing is less costly in terms of both time and money, and we rigorously review our PO’s requirements and display designs before we move on to development. We identify the points that need to be corrected or that we think are missing, and make sure that they are corrected, and then we move on to the development phase. In this way, I can say that we use static testing techniques intensively. So things are tested on documents before they go into development. Because we know that the cost of errors noticed during development and testing is always higher. I will describe the testing techniques associated with this item in a different article. :)
Error costs
- Identification of Requirements — Analysis Phase 1€
- Design Phase — 10€
- Development Phase- 100€
- Test Phase- 1000€
4. Defects cluster together
Errors are concentrated in certain areas of the software. Research shows that bugs in software are concentrated at a certain point. Considering this situation, it has been observed that errors can be detected faster in the tests performed. These error sets can be used in risk analysis. And test activities can be determined accordingly. We can observe that the errors are mostly concentrated where there are definitions such as <, >, <=, >=.
I want to give a real example to make this article more understandable. Our team has a structure called automatic pricing in the domain. Automatic pricing in its simplest form; allows the sellers to determine the conditions they want in order for the products to win the buy-box, and allows the price changes to change automatically according to these rules, thus enabling the sellers to be included in the competition more easily. In addition to being active and passive, we have added the state of being deactivated to these defined rules. In cases where the defined rule can no longer be changed, the rule is deactivated. Thus, vendors can identify which rule is disabled, easily update their rules and get involved in this process again. The minimum and maximum price range is determined in the defined rule. Here, we have given the definitions of <= and >= as < and >. And in this case, when the rule should have been active when the minimum or maximum price was equal to the buy box price, it caused an error and caused the status to be deactivated. If we bring back item 4 of the 7 principles of testing, we can easily spot bugs early on before our app goes live.
5. Beware of the pesticide paradox
The pesticide Paradox principle says that if the same set of test cases is executed again and again over a period of time then these sets of tests are not capable enough to identify new defects in the system.
If you are constantly testing the same module with the same cases and scenarios for a long time, it is possible that you will not find any bugs in these modules after a while. That’s why it’s always recommended to improve the quality of test cases and change test cases over time with product changes. However, even after updating the test cases regularly, you cannot say that the test is 100% complete. Existing test data needs to be updated and new test cases added to find new bugs. However, the importance of these test scenarios should not be forgotten. It can be used in regression tests.
6. Testing is context dependent
Software testing varies according to project context and conditions. Each type of software system is tested differently. According to this principle, the test depends on the content of the developed software and this is completely true. The truth is, every app has its own requirements, so we can’t just put the test in a box. Of course, every application goes through a defined testing process, but the testing approach may vary by application type. Various methodologies, techniques, and types of testing are used depending on the nature of an application. For example, e-commerce applications and health applications tests are different from each other. It has to be different. Because the requirements and process structure of the two applications are different from each other.
7. Absence of errors is a fallacy
We can’t find any new bugs, it can create the illusion that we are doing successful software. After the test is successfully completed, it is thought that there will be no malfunctions in the product and that it will fully meet the needs of the customer. However, as we will remember from Principles 1 and 2, it is impossible to test software 100% and the important thing is to detect bugs.
Conclusion
At Trendyol, we always consider the 7 principles of software testing in our development and testing processes. We never forget the importance of early testing, that it is not possible to test the software 100%, the important thing is to know the risks, to develop our testing strategies accordingly and to implement them with as few errors as possible. Adopting 7 principles helps us produce high quality products and determine our overall testing strategy.
Resources
https://www.turkishtestingboard.org/files/FL-Syllabus-2018-GA.pdf
