Software quality assurance and testing are major parts of software development. It allows the developer to verify that the software works as expected. According to the effort distribution of phases of the Software Development Life Cycle (SDLC), testing requires almost as much effort as development/coding, with some sources allocating more percentage to testing than development.
To understand better the meaning and relevance of testing, we’ll look at a little history on quality assurance going as far back as the Quality revolution in 1940. I found this information interesting because it shows that the concept of software testing is deep-rooted and gives more context to the “why” of it.
The Quality revolution was started in Japan in the 1940s. In the 1950s, W. Edwards Deming, one of the founding fathers of the Quality revolution introduced Statistical quality control (SQC) to Japanese Engineers. The SQC mainly involved the use of statistics and measurements to determine quality. Deming went on to introduce the Shewhart cycle which involved four main processes Plan, Do, Check and Act (PDCA).
In 1954, Joseph M. Juran, another of the founders of the Quality revolution, began the move from SQC to Total Quality Management (TQM).
The summary of TQM is
- Quality takes priority, not short-term profits
- Decisions are driven by facts and data
- The customer comes first, not the producer
- Employees participate in ensuring the quality of the product
These views have since been adopted by countries around the World, such as the USA, to improve the quality of products.
The Quality revolution opened the floor for more intentional conversations on quality across fields. The software Capability Maturity Model (CMM) architecture developed at the Software Engineering Institute is based on the principles of product quality that have been developed by Deming, et al.
The Five Views of Software Quality
In Software engineering, when we talk about quality, it is important to note that quality means different things to different people. People have different perceptions of Software quality. Which is why when one of the things we must understand are the five views of Software quality, which are:
- The Transcendental view
- The User view
- The Manufacturing view
- The Product view
- The Value-based view
The Transcendental view: The word transcendental has different meanings, but they all point towards an experience, event, or idea that is unique, abstract, extravagant or not ordinary. The use of the word transcendental to describe software quality describes this view as one that can be known or realized through experience but is difficult to define.
The transcendental view tells us that quality stands out, and when seen or experienced, can be recognized.
The User View: The user view defines quality by how much the product satisfies the user’s needs. Quality here is personalized to the user and is determined by its fitness for purpose.
The Manufacturing view: In the Manufacturing view, quality is reciprocal to the extent to which the product meets its requirements or specifications. The manufacturer looks at this product and is able to rate its quality by comparing the specifications to the outcome of the product/project.
The Product View: This view ties quality with the inherent properties of the product. It assumes that if a product has good internal characteristics, then it would have good external characteristics. For instance, a lot of people who buy laptops, buy them based on this view — They consider the internal properties such as the RAM, storage, processing capacity, and so on. These properties help them determine the quality of the product they are purchasing.
The Value-based view: The value-based view attributes quality to how much a customer is willing to pay for it. It ties quality to the worth of the product.
There have been various quality models created to define quality and its attributes. Two of the most notable are the International organization of Standardization (ISO) 9126 and the Capability Maturity Model (CMM). The ISO 9126 quality model defines quality based on six general characteristics: functionality, reliability, usability, efficiency, maintainability, and portability. The CMM framework establishes five levels of maturity for a development process namely Initial, Managed, Defined, Quantitatively Managed, and Optimising. The levels follow an increasingly organised structure of the organisation’s processes.
Testing
Software testing is important in determining the quality of a software product. We are able to contentiously improve our software quality through a cycle of testing, finding and fixing bugs.
Software quality assessment is broadly divided into two categories — Static analysis and Dynamic analysis.
Static Analysis: This testing process involves reviewing documents related to the system, such as requirement documents, design documents and source code without executing the program. It examines the code structure and the reasons behind the behaviours of the different features (or bugs) that may arise during the running of the program.
Dynamic Analysis: This involves testing and evaluating the program while it is running. It aims to find possible systems failures in the process. It checks if the behaviour of the system during runtime matches the expected behaviour.
Static and Dynamic tests go hand in hand and should be carried out repeatedly to detect as many faults as possible and fix them in the early stages of the development process.
Verification and Validation
Verification and validation are concepts related to software testing which involve a set of activities aimed at checking that the software satisfies its specifications and the purpose for which it was created.
Verification is the process of checking that the system conforms to the requirements and specifications established before the implementation of said system. These specifications can be in the form of the requirement document, design document or even the source code. This process involves activities such as Inspection (of code and other documents), Walk-throughs, and other Static analysis activities.
Validation is the process of checking that the system meets its intended use. In other words, validation ensures that the product is what the customers expect, and it does all that it was created to do. Validation activities involve reviewing the program during runtime and testing and other dynamic analysis techniques.
Summarily, Verification answers the question
Are we building the system correctly?
While Validation answers the question
Are we building the correct product?
FAILURE, ERROR, FAULT, AND DEFECT
These are some key terms used in testing. They are similar terms but have notable differences.
Failure: This refers to a situation where the system’s external behaviour does not fulfil the requirements of the system. Failure is realized after the production stage of development.
Error: This is a human mistake that occurs during development. It is a state of the system which could lead to failure if not corrected.
Fault: A Fault is an unintended behaviour of a system. A fault causes warnings and can go unnoticed for a while until activated by an event. In which case it results in an error state.
Defect: Also known as bug is an issue or error that prevents the system or a feature within it from performing as expected.
The objectives of Testing
The objectives of testing are to
- Determine what does not work
- Determine what works
- Reduce the risk of failures
- Reduce the cost of testing
Levels of Testing
The general levels of system testing are
- Unit testing
- Integration testing
- System testing
- Acceptance testing
Unit testing involves testing a program unit, method or modules in isolation.
Integration testing involves assembling or grouping modules to form larger subsystems that are tested. The purpose of this test is to expose faults in the interaction and communication of these modules when they are integrated in the actual system environment.
System testing involves a wide spectrum of tests conducted on a fully integrated system. It involves testing the complete system and includes activities such as functionality testing, stress testing, robustness testing, and so on.
Acceptance testing is carried out by the customer. After the developers have completed all their tests, the customer does the acceptance testing. This test is more about the quality of the system and less about finding bugs or faults. The criteria for this test are the customer's expectation of the system.
The two types of Acceptance tests are User Acceptance Test (UAT) and Business Acceptance Test (BAT). User Acceptance Test is conducted by the customer to ensure that the system satisfies the agreed requirement specifications between the customer and the development team. Business Acceptance testing serves as a prelude procedure to UAT; it is conducted by the customer in the presence of the development team to verify that the system will pass the user acceptance test.
Flutter Testing Levels
In Flutter the major test levels are
- Unit Test
- Widget Test
- Integration Test
Unit test: Similar to the above definition, it involves testing individual units, methods or classes.
Widget Test: This test involves testing individual widgets; it ensures that the widgets look and behave as expected upon user interaction with them.
Integration Test: This test is the equivalent of a system test as defined above. It involves a complete test of the app or a group of app widgets and how they interact and communicate with one another including the system services. This test is visual and runs on a physical device or simulator.
In subsequent articles, we will be discussing and demonstrating how to run the different tests in Flutter.
That’s a wrap on this Software quality assurance and testing article. I hope this article has given you a better idea of software quality assurance and testing.