Empowering Performance Testing: Introducing State-of-the-Art Framework
Introduction
In today’s fast-paced digital landscape, ensuring optimal application performance is paramount for us at Airtel to deliver seamless user experiences and maintain a competitive edge.
We at Airtel have nearly 18 million devices connected on our Airtel IoT platform. Our Airtel IoT (Internet of Things) holds immense significance in today’s interconnected world by playing a pivotal role in enabling seamless connectivity and data exchange between devices, systems, and applications. It empowers us to monitor, control, and optimise our operations, leading to improved efficiency, productivity, and cost-effectiveness. By leveraging Airtel’s robust infrastructure, secure network, and scalable solutions, we can unlock the potential of IoT to enhance our decision-making, automate processes, and create innovative services. The significance of Airtel IoT lies in our ability to drive digital transformation, enable smart solutions, and fuel the growth of industries across sectors, ultimately shaping the future of connected technologies. Hence, it is very important for us to keep the high performance of our IoT platform to provide a seamless experience to our customers.
It’s crucial to locate performance bottlenecks in order to ensure high scalability and throughput, and to guarantee that each order will be processed in the shortest amount of time possible with 0 failures (or to maintain 99.99% success rate).
In this blog, we will take you on a step-by-step journey through our Performance Testing Framework and highlight its unique features and capabilities.
Background and Motivation
Before the advent of Performance Testing Framework, we faced numerous challenges when it came to effectively conducting performance tests. Traditional approaches lacked scalability, customisation options, and real-time monitoring capabilities, which hindered our ability to identify performance bottlenecks and optimise system performance efficiently.
Existing tools and processes often fell short in meeting our evolving performance needs. As our applications grew in complexity and user demand surged, we realised the pressing need for a comprehensive and adaptable framework that could keep pace with our expanding requirements.
Key Metrics Helps to Develop Performance Testing Framework
- Key Performance Indicator(KPI) to achieve performance benchmarks
Let’s Dive into the Framework Overview
The Performance Testing Framework is a comprehensive and innovative solution designed to empower organisations in conducting efficient and insightful performance tests.
The Core Framework capabilities :
- Scalability — Assisting in assessing and planning for future growth and scalability.
- Customisation — Use modular approach for future customisation with ease.
- Low Code development — Ease of scripting.
- Time Saving — Automates test execution, and provide single click execution which remove dependency of performance testing for day to day execution.
- Enables Real-Time Monitoring — Real time monitoring of key performance metrics to access health and system performance.
- Robust data visualisation and Reporting capabilities — Presenting performance data and insights in a clear and meaningful way through dashboards, charts, reports.
- Diagnostics & TroubleShooting — Integrate tools and techniques which helps in detailed profiling, tracing & debugging capabilities to identify the root cause.
- Integration & Automation — Seamlessly integrate CI-CD Pipeline and can be used with different applications.
- Collaboration & Communication — Facilitating collaboration among development, testing, product & operation to effectively address performance issue. Also for every execution it mails the detailed report to all the stakeholders.
Let’s explore the key components and features of the framework
- Jenkins Job Creation: The framework integrates with Jenkins, a popular continuous integration tool, to trigger performance tests. By creating a Jenkins job, users can easily initiate performance tests with just a few clicks. This streamlined process eliminates the need for manual intervention and ensures consistent test execution.
- Test Configuration: With this framework, users have full control over test configuration. From the Jenkins job, users can select the appropriate JMeter script, define the number of users, set the ramp-up period, specify the test duration, configure TPS (Transactions Per Second), and determine the loop count. This flexibility allows testers to tailor the tests to some specific scenarios and accurately simulate real-world conditions.
- Load Generation: Efficient load generation is a key aspect of performance testing. The framework employs a shell script that executes the selected JMeter script on the load generator, generating the desired workload. This streamlined load generation process and ensures optimal resource utilisation and reliable test results.
- Server Monitoring: To gain comprehensive insights into the system health during performance tests, the framework incorporates server monitoring. We utilise a server agent that collects vital performance metrics from the servers under test. These metrics include CPU usage, memory utilisation, disk I/O, and network activity. Real-time monitoring of server metrics enables quick identification of performance bottlenecks and helps pinpoint their root causes.
- Metrics Visualisation: The framework leverages InfluxDB, a powerful time-series database, to store dynamic JMeter data generated during performance tests. To visualise this data in real-time, we utilise Grafana, a flexible and feature-rich dashboarding tool. With Grafana, testers can monitor key performance indicators, identify trends, and gain valuable insights into application performance during the test execution phase.
- Test Reports and Logs: After completing a performance test, the framework generates comprehensive reports. These reports include client-side performance metrics, system health status, and any identified performance bottlenecks. The reports provide the stakeholders with a clear understanding of the application’s performance characteristics, allowing them to have an informed decision-making. Additionally, all test logs are saved and easily accessible within the generated reports.
- Stakeholder Communication: Effective communication is vital in performance testing. The framework automatically sends email notifications to the stakeholders, providing them with the generated reports and important insights. This ensures that all relevant team members stay informed about the test results and can take appropriate actions, based on the findings.
By incorporating these components and features, the Performance Testing Framework provides a seamless and efficient testing environment. It enables organisations to conduct performance tests with precision, uncover performance bottlenecks, and ultimately optimise application’s performance to deliver exceptional user experiences.
Use Cases and Examples
Challenge
Before leveraging the Performance Testing Framework, our application struggled to handle high user loads. With a single digit Transaction Per Second (TPS) and an alarming max response time i.e. >30seconds, our application faced severe performance bottlenecks that negatively impacted the user experience. It was clear that significant optimisations were necessary to ensure acceptable performance levels.
Pre-Requisitie Setup
Integrate App Dynamics — It’s an Application Performance Management(APM) tool which helps in monitoring the app infrastructure, along with that it provides the code level visibility to identify who is taking extra time during the execution
AWR (Automatic Workload Repository) — AWR captures the Oracle database activity, it has many different sections with a large amount of performance data. You can use this information to compare statistics captured during a period of bad performance to a baseline, and diagnose performance issues.
Solution :
Leveraging the Performance Testing Framework, we conducted a series of performance tests to uncover the root cause of the high response time and to identify areas for optimisation. Framework seamlessly integrated with App Dynamics, a powerful application performance monitoring tool, allowing us to gain deep insights into the application’s behaviour and performance.
Through analysis, we identified several areas that required attention. The performance tests revealed SQL queries with poor execution plans, inefficient Java methods and code, and suboptimal Oracle database configurations. Armed with this knowledge, we initiated a systematic optimisation process.
Results
The impact of our performance optimisation efforts was remarkable. By leveraging our Performance Testing Framework, conducting thorough performance tests, and implementing targeted optimisations, we achieved outstanding results:
- Transaction Per Second (TPS): We were able to support a significant increase in TPS from a mere x to a robust 7x TPS, showcasing the enhanced scalability and performance of our application.
- Response Time Improvement: The response time of the application decreased dramatically to less than 200 milliseconds. This improvement ensured a highly responsive user experience and satisfied user expectations.
Conclusion
A Performance Testing Framework addresses the limitations of manual performance testing, empowering testers to efficiently evaluate and enhance application performance. By automating test execution, integrating with tools like JMeter, Jenkins, GIT HUB, InfluxDB, and Grafana, and providing comprehensive monitoring and reporting capabilities, the framework enables organisations to proactively tackle performance issues, optimise system behavior, and deliver exceptional user experiences.
Coming up next:
- Detail Framework Implementation
- GC Logs Analysis through Jmeter
Like coal become diamond in high pressure, performance testing produces scalable applications.
Special Thanks: IOT performance team at Airtel including Sagir Ansari, Hitesh Panda, and Ankit Goel(HOE) for continuous support and guidance.
Blog Authors : Saurabh Gupta & Sagir Ansari
