10,000 Ft’ Overview of Performance Testing

In this write up we will provide a high level view of different types of performance testing methodologies and terminologies. If you are puzzled about the need of a performance testing strategy for your application then please go through our article “7 Reasons WHY Performance Testing Is Paramount”. Lets get started on the 10,000 ft’ overview..

Types of Performance Testing :

1)Load Testing : In load testing the application/system goes through a gradual increase of users as well as number of transactions to reach a target load. The main purpose of load testing is to find out any bottlenecks in the application and observe response time , through put etc. under a desired load.

2)Stress Testing : Understanding the maximum processing threshold of a component keeping a close tab on SLA is a major requirement of performance testing which is achieved through the technique of stress testing. It also referred as break-point testing. Stress testing primarily helps in capacity planning of various components in an application.

3)Soak Testing : Soak testing is also termed as endurance testing. In soak testing an applications goes through a predefined load pattern for elaborate test duration without interruption as expected in production environment. It highlights endurance of the application components over a period of time.

4)Spike Testing : As the name suggests, it simulates sudden burst of traffic situation in the application. Its an advanced technique of performance testing which goes beyond the first three categories as discussed above. The primary purpose of the spike testing to evaluate robustness of an application. E.g. inducing an overwhelming volume of traffic for an application to create a scenario of denial of service and observe the system to learn more on the strength/weakness of the application.

5)FMEA Testing : Its also another advanced performance testing technique which primarily helps to find out the factors impacting application resiliency. In this technique the behavior of the primary components is observed under high load while its dependencies are made to go offline abruptly and bring those back online. This approach assesses the fail over scenarios and recovery patterns thoroughly.

Performance Testing and Engineering Terminologies:

• NFRs : Non-functional-requirements stand for the operational requirements of the application which is very different from the functional work flow. For e.g. performance requirements such as processing time, processing capacity etc., security requirements such as user-id & password compliance etc.
• Performance Engineering : Performance testing activities starts at the later part of development or after initial phases of functional testing where as performance engineering activities start with the very first step of the design phase to ensure compliance with the NFRs and create a vision for application performance.
• Performance Testing Strategy : A detailed documentation having performance objectives with clearly defined scope, approach, deliverable, schedule, limitations, risks, data , test environment requirements.
• Scalability : The degree to which system processing capacity can be widened either by adding hardware resources (i.e. vertical scaling) or by adding more instances of server nodes (i.e. horizontal scaling).
• Reliability : The consistency of the application to be able to produce same result under a given load for a specific time period.
• Availability : It simply stands for the up-time of the application.
• Robustness : It means the strength of the system to withstand certain types of failures.
• Resiliency : The degree of recoverability for a system to bounce back from various type of failures.
• KPI : Key performance indicators are the set of key metrics that help in detailed analysis of application performance.
• SLI : A few critical KPIs form the set of service level indicators those are most valuable for assessing system performance.
• SLO : Service level objective is a very precise quantifiable target of system performance. SLO is calculated based on the measurement of SLI values.
• SLA : Service level agreement is a contract between the application owner and customer where the customer is promised to be provided with a certain service in a given condition.
• TPS : Transactions per second, a measurement scale for traffic volume.
• Regression Flows : The 20% base set of transactions driving mostly 80% of the application volume. This is also called user journey.
• WLA : Work load analysis is an exercise to narrow down the exact mix of transactions with corresponding TPS. It also focuses on the various data sizes i.e. request / response packets as part of the analysis.
• Virtual User : For performance test execution virtual users are created with sample test data to help simulating the actions of real users.
• Pacing Time : The time difference or delay between script/transaction iteration.
• Think Time : The time difference or delay between consecutive transaction or steps in the script.
• Ramp Up Time : Total time spent in achieving a given peak load with a systematic increase in traffic volume.
• Ramp Down Time : Total time spent in getting down from peak traffic to no/zero traffic with a systematic decrease in traffic volume.
• Concurrency : Simultaneous users or user sessions in the system driving the user load or overall set of transactions.
• MTTR & MTTF : Mean time to repair and Mean time to failure are key metrics for measuring reliability. Mean time to repair is the time taken to repair a system and bring it back online from a failure. Mean time to failure is the time between a system’s availability to un-available status.
• Shift Left : Shift left is the fundamental concept of performance engineering to detect performance bottlenecks in the application during initial phase of SDLC. Core theme of shift left is to test early and test often.
• Single User Profiling : Investigating bottle neck in the application while running a single user. This exercise is a predominant methodology of application profiling to evaluate the time to process specific blocks of application code.
• Front-end-profiling : This is a single user profiling technique to find out the response time break up at front end of the application. e.g. time spent in executing java script, loading images , applying themes etc. , time spent in the device for mobile apps.
• Server-side-profiling : This is also a single user profiling technique to narrow down time spent on the server side processing . i.e. application server processing indicating the block of code, data base processing showing the long running queries , host system processing etc.

We have discussed the key terminologies of performance testing in this article while touching up on performance engineering. Please keep me posted with your comments/suggestions/questions.In the upcoming post we will discuss about “6 Steps To Create Performance Testing Strategy”.