What Is Performance Testing?
Performance testing is a non-functional testing technique that exercises a system and then, measures, validates and verifies the response time, stability, scalability, speed, and reliability of the system in production-like environment.
It is a process in which we try to find the bottleneck for our application and its certain features, i.e at what particular stress point our web application will crash.
Different applications may have different performance benchmarks before the product is made available to the end user. A wide range of Performance tests are done to ensure that the software meets those standards. Suppose you own an e-commerce website, and you want it to be robust, stable, fast and 24/7 online because you want to remain in the business as well as on the top of it. Initially, it wouldn’t be a big deal but when your customer base will increase there will be a lot of loads on your website. And it will crash if it’s not properly made and tested before release and the company will bear a huge loss in business as well as customer trust. So, to prevent this performance testing is used.
This article is focused on some commonly used Performance testing techniques and the process involved in testing an application.
Performance Testing Metrics
Performance metrics are used to calculate the performance parameters and find out the areas of the application which are weak and creating bottlenecks in the application.
Metrics are used to track the project progress. It is used to make a baseline for all the tests. Using metrics, we can measure the main cause of the problem. Using metrics, we can compare the result of the different tests and find out the impact of any changes made to the application. It helps in improvement of quality of product. It provides clear output of the activity and find out the areas which require attention.
There are numerous performance metrics generated during performance testing, but we cannot evaluate all the metrics. I try to select Some of the commonly used Performance Metrics are described below.
- Response time: Response Time is the time elapsed between a request and the completion of request by the system. Response time is critical in Real Time Applications. Testers usually monitor Average Response Time, Peak Response Time, Time to first byte, Time to last byte etc.
- Latency: Latency is waiting time for request processing. It is time elapsed between request till the time first byte is received.
- Error Rate: Error rate is percentage of requests resulting in errors as compared to total number of requests. Rate of errors may increase when application starts reaching its threshold limit or goes beyond.
- Monitoring Transactions: Transactions are executed for users to test its performance. It is important to know the status and response time of each transaction happening through the application. The average response time of transactions is noticed for every single transaction to evaluate the online user’s impact on the system. The average response time should not be more than 3 seconds.
- Bandwidth: Bandwidth is an important metrics to check network performances. It is the measure of Volume of data per second.
- CPU Utilization: This is a key metrics which measures the percentage of time CPU spends in handling a process. Ideally the CPU utilization should not be more than 70 %.
- Memory Usage: This metric gives information on free disk space, memory used during execution of a scenario. Disk Space provides information of disk space used.
- Pass failed transactions: It is important to have idea about how many transactions passed and how many transactions failed during performance Testing
- HTTPS response per second: The HTTP response graph tells us about all the HTTP status codes generated every second during the performance testing. It tells us about the status on web server on every second of performance Testing.
- Connections Per Second: This performance test metric tells us about the number of TCP/IP connections opened and shut every second during performance Testing. The connections per second should always be lesser than the number of hit per seconds as it is very costly to open and shut TCP/IP connections.
Performance Tests Process
Performance testing process involves certain basic steps as depicted in following diagram.
Identification of Test Environment
The first step in Performance testing involves identification of testing environment which includes hardware, software, network configuration, database setup etc. In a perfect world scenario, it is ideal to test in target production environment. But if that is not possible then intermediate testing can be done at a simulated environment same as target environment. Though this may be a costly option. If cost is a constraint, then a subset environment with similar or proportionally lower specifications can be set up for testing purposes.
Determination of Performance Criteria
Every application has its own performance criteria which must be identified during requirement specification stage of SDLC. The testing metrics are based on these criteria. Those metrics may include response time, resource utilization, throughput etc.
Test Cases Designing
Test cases must be designed keeping in mind that software will be launched in a real world to be used by different users with different style of usage. The cases should include excessive loads, spikes, and large data volumes. It is vital to plan for all possible scenarios for comprehensive testing.
Configuration of Test Environment
Test environment framework must be configured with relevant software, hardware, third party softwares, network setup, database setup, testing tools, debugging tools etc. It is important to set up the environment as per the guidelines because any issues here may have adverse effect on testing results, rendering the whole process unproductive.
Test Execution
Execute the test cases as per the test plan, monitor and record the results.
Analysis and Reporting
Test reports include conformance to requirements (or baselines) as well as secondary metrics on system resource utilization, e.g., CPU, memory, and disk.
Fine Tuning and Retest
Based on test reports, the software can be fine-tuned by the development team to perform better. Once the changes are done and redeployed, the test execution is done again followed by analysis and reporting of results. This cyclic process continues till the margin is within permissible limits as per the stakeholder rrequirements.
Before marketing your software product, it is necessary to do Performance testing. It is most important to understand the complete performance requirements of product before starting your testing. In this article we learned about what all performance testing metrics needs to be considered.
I hope this article is useful to understand how to start with Performance Testing and what metrics to use.
