Detailed Guide to Jmeter Performance Testing
Performance Testing
Types ofNon Functional Testing :- Performance Testing , Security , localization, Usability ,scalability , Compatibility testing etc.
Performance Testing :- Performance testing is mainly of 5 types :-
- Load testing — checks the application’s ability to perform operations with maximum load on the server . The objective is to identify maximum requests a software / product can sustain without compromising its functionality.
- Stress testing — involves testing an application with more than load count .The objective is to identify the breaking point of an application.
Classic example of Load and Stress Testing -
- Endurance testing — is done to check the duration of load a software can handle over a long period of time without breaking the system.
- Spike testing — tests the software’s reaction to sudden large spikes in the load generated by users.
- Scalability testing — The objective of scalability testing is to determine the software application’s effectiveness in “scaling up” to support an increase in user load. It helps plan capacity additions to your software system.
Performance Testing Process and Flow diagram :-
The process of performing performance testing differs widely but the underlying objective to identify the potential or durability of software .
In order to achieve effective results , we need to consider below points before applying load on application
- Role of Test Bed / Environment :
Test Environment is one of the vital components while or before performing Performance testing of our application .The Test Environment should be very close or mirror of the production environment for Load testing . This is to make sure all the instances , CPU , Memory , bandwidth are under the same load when load is applied during the test .Any mismatch between configuration will not yield good or accurate results.
- Understand the End goal :
Once the Test bed is ready , User need to focus on finding the end goal or expected results . Based on the end goal , Identify the request count using Equivalence partitioning and boundary value analysis logic.
- Create Test Plan : -
Prepare the test plan which will consist of request count vs ramp time ( the time to complete the execution ) . Test plan should be created at least for 10–15 rounds to analyse the application behaviour under different scenarios.
- Design Application to the Tool :-
Once identification of test data / request is completed , Design and configure your application properties in Performance testing tools such as load runner , JMeter , locust, Blaze meter etc .
- Execution and Analysis of results :-
Post successful designing in any tool , execute the test plan with lower count of request and raise the count keeping the ramp up time constant .
Monitor the behaviour of application with every run .Check various parameters such as Latency, throughput , start time , stop time etc after execution of every test set
Commonly available tools in the market for Performance Testing :
- JMeter :- JMeter is an open source tool which is one of the widely used tools for performance testing . It’s a tool developed by Apache and supports testing different kinds of applications ranging from HTTP/HTTPS-based applications to Database and FTP servers.
2.Load Runner : This tool is capable of simulating hundreds of thousands of users, putting applications under real-life loads to determine their behaviour under expected loads
3. Blazemeter : BlazeMeter is a SaaS-based performance testing tool, fully compatible with JMeter and many other open source load testing tools.
4. Locust : Locust is an open-source load testing tool that gauges the number of concurrent users a system can handle. We can write simple behaviors using Python.
5.Load Ninja :- This cloud-based load testing tool empowers teams to record & instantly playback comprehensive load tests, without complex dynamic correlation & run these load tests in real browsers at scale.
Download and Install Jmeter for Mac Users : -
- Navigate to official website of Apache.org and then navigate to JMeter project by clicking on Apache Jmeter
- Navigate to Download section for mac users by clicking on Download Jmeter (Mac Users )
- Under the Binaries section , click on apache-jmeter-5.5.zip sha512 pgp
- Unzip the JMeter binary to a local directory
- Or another way of installing Jmeter package in our mac is by using brew
Command :- brew install jmeter
- Once unzipping is completed , Package folder and its root directory will look like below
- We can use launch the application by running below command from bin folder of root directory
Command to launch Application :- sh jmeter.sh
- In case to stop jmeter server , Run the Shutdown client to stop a non-GUI instance gracefully and abruptly
Command 1 :- sh shutdown.sh ~~ gracefully
Command 2 :- sh stoptest.sh ~~ abruptly
Elements / Components of a Jmeter :-
- Thread Group :-
Thread group refers to collection or group of threads. Each thread is a request / user to an application under test. If the count of the thread group is ‘X’ ,JMeter will create and simulate ‘X’ user requests to the server under test to determine the performance of an application.
User can access the Thread group Element in JMeter by Right Click on Test Plan-> Hover over Threads-> Click Thread Group
Thread Group properties :-
Once user select the ‘Thread Group’ option in the test plan , we need to understand three vital characteristics of Thread Group in JMeter
- Setting the number of threads :- This implies the number of threads / user sending the request to the application or server.
- Setting the ramp-up time :- Ramp-up time is the amount of time to complete all the thread counts. If the thread count is 10 and ramp up time defined is 20 , it means for every 2 seconds ,a new user request will be added / sent to the application server.
- Setting the number of test iterations :- Iterations or simply called as loop , that defines number of iterations of thread count for a defined ramp up time
Below is the sample glimpse of Thread group in JMeter : -
2.SAMPLERS :-
JMeter has one of the basic controllers which is known as ‘Samplers’. Samplers are used to send different types of requests to the server. Jmeter supports a variety of samplers for load testing . The most commonly used sampler is HTTP Request sampler that can be used to perform CRUD operations such as POST, PUT GET, or DELETE on a HTTP service.
Few other useful samplers are −
- FTP Request
- JDBC Request
- Java Request
- SOAP/XML Request
- RPC Requests
We can launch the Samplers in JMeter by following the path-Right Click on Thread Group -> Hover over Add -> Hover Over Sampler -> Click on the required Sampler
Below is the sample snapshot of Sampler in JMeter
3. LISTENERS
JMeter Listeners are the test plan elements that are used to view and analyze the result of performance tests either in tabular or graphical form. They also provide the different response time matrices (average time, minimum time, max time, etc) of a Sampler request.
JMeter provides a wide range of capabilities for listeners to analyse the result set.The following list consists of all the Listeners JMeter provides −
- Sample Result Save Configuration
- Graph Full Results
- Graph Results
- Assertion Results
- View Results Tree
- Aggregate Report
- View Results in Table
- Monitor Results
- Distribution Graph (alpha)
- Aggregate Graph
- BeanShell Listener
- Summary Report
User can configure / add listener in Test plan by Right Click on Test plan -> Hover over Add -> Hover Over Listener -> Click on the required Listener
Below is the sample steps to add a Listener in JMeter Test Plan:-
4. Important Config Elements:-
4.a. Header Manager :-
Header manager is used to add all predefined headers of the request . JMeter provides an ‘HTTP Header Manager’ config element to attach that additional information along with the request. JMeter supports multiple Header Managers at different levels.
Below is the sample steps to add a Header Manager in JMeter Test Plan:-
4.b. Authorization Manager :-
JMeter provides the HTTP Authorization Manager config element to use credentials for authentication. With this component, we can give credentials for several domains and authentication protocols.
Below is the sample steps to add a Authorization Manager in JMeter Test Plan:
5. Assertions in JMeter :-
JMeter allows users to assert the expected results with actual one .Assertions in JMeter are the test plan elements that are used to validate the response received from server for a particular sampler request.
Below is the sample steps to add a Assertions in JMeter Test Plan:
The most commonly used Assertions in Jmeter are :-
5a. Response Assertion
The response assertion used in test scripts to validate a pattern in the response body, header, code, message etc. There are different pattern matching rules to validate the response like-
- Contains — If the response text contains the regular expression to be matched
- Matches — If the whole response text matches the regular expression
- Equals — If the whole response text matches the pattern(not regular expression but the pattern string)
- Substring — If the response text contains the pattern(not regular expression)
- Not — To check that the pattern is not present in the response text
5b. Duration Size :-The duration assertion is used to validate that the sampler request gets processed within a specified amount of time. It limits the time for all the responses.
5c. Size Assertion :-The size assertion is used to validate the size of the response with a specified value in bytes.
NOTE : In Order to view the outcome of these assertions, JMeter has provided one Listener known as “Assertion Results”
Assertion Listener compares expected outcome vs received outcome either at status code or response / text level.
Assertion Results will look like below post execution of Test Plan
Execution of Tests in JMeter
We will see two major types of Performance Test using the JMeter tool i.e.
a. Rest API performance Testing Using Jmeter :
Before any type of execution , we need to provide basic configurations and Elements in Jmeter as per Rest API behaviour and its specifications.
All the Configuration and Specification such as Thread group, samplers, Listeners, Assertions etc is already explained above.
Following are the steps and details of a Rest API Test Plan :
- Defined Thread count = 50 Users , ramp up period = 10 seconds and Loop count = 1
Below is the snapshot of ThreadGroup of my Test plan for Rest API:
- Select HTTP Sampler and Select type of request i.e. GET, POST, PUT ,DELETE etc. and pass the Server path or endpoint . Do pass port number if available
- In case Rest API requires body data to be passed , then add the request in Json format
Below is the snapshot of Sampler of my Test plan for Rest API:
Add the Table Listener to view / analyse the Latency , sample time (ms) , send/ received bytes and thread status .
Below is the snapshot of Table view Listener of a given Thread run
Graph Results Listener
For better visualisation for results , we can use Graph Results Listener.
At the bottom of the picture, there are the following statistics, represented in colors:
- Black: The total number of current samples sent.
- Blue: The current average of all samples sent.
- Red: The current standard deviation.
- Green: Throughput rate that represents the number of requests per minute the server handled
Few Key points to consider while evaluating the results:
a. Deviation value should be lesser always to consider it for good performance
b. Throughput should be higher that means number of requests processed per minute/ second , should be ideal for good performance
NOTE : Results depend on several factors like Test Bed such instance size , memory , internet speed, CPU power etc. Hence, it’s very unlikely that you will get the same results always .
Conclusion:
If you set this up correctly, all you need to do is make changes to your Thread Group , modify the sampler details , Config manager and include the Listener to analyse the result.
References:-
- https://jmeter.apache.org/
- https://github.com/apache/jmeter
- https://www.blazemeter.com/blog/jmeter-tutorial
Thanks !
