In this post we will present you a step-by-step approach to formulate performance testing strategy. If you still have doubts regarding the need of performance testing then don’t hesitate to check our article 7 Reasons WHY Performance Testing Is Paramount. Alright folks, lets get started with this one.
Step-1 (Define Scope & Criteria) :
Clearly mention the applications and corresponding transactions to be performance tested. In order to narrow down the transactions to be included as part of performance testing, follow the thumb rule of including regression flow. Also take feed back from development/design team, business/product team etc. to keep a close watch on the transactions those are impacted as part of any new code change or new feature launch. Its very important to mention the list of deliverable as expected out of the performance testing exercise ; such as a detailed test strategy, certification report, tuning configuration parameters etc. It is very critical to identify the types of performance testing required to give a go ahead for production deployment. Performance exit criteria is also driven by the NFR document, so please make sure to refer it back while giving a go ahead for production deployment.
Step-2 (Configure Test Environment) :
Performance test environment configuration is a very important step in performance test strategy which needs to be done with utmost attention. It is ideal to prepare the test environment on par with production environment but most of the times its not possible due to several reasons such as limitation in budget, support resourcing etc. So the idea is to prepare the test environment in proportion to production capacity by scaling it down i.e. 50% of production or 30 % of production or 10% of production etc. as feasible. Once the hardware resources are provisioned the next step is to ensure the versions of the various software components such as operating system, application Server, webserver, data base servers etc. are deployed appropriately by configuring those in synch with planned production versions. Also ensure that, the latest version of the application component as planned to go live is getting deployed rather than any other old version with required tunable parameters. Monitoring tools help in analyzing the test results which is setup in this step as needed.
Step-3 (Test Plan & Design) :
This step is about planning and designing the test scenarios based on the test scope and exit criteria. As far as planning is concerned it consists of preparing a detailed work load strategy with help of WLA ; which will be used as an input for creating test scenarios. Test plan also highlights a detailed test schedule with appropriate time window for component testing, baseline testing, SOAK testing etc. as needed with the end date of test closure and certification result announcement. In this step, the required test data is prepared based on the traffic requirements as given in the work load strategy. Performance testers are assigned to create test scripts for the specific flows/transactions with the given data and also prepare test scenarios with inputs such as appropriate ramp up/ramp down , think time , pacing , numbers of virtual users etc.
Step-4 (Test Execution) :
The very first step of test executions is to run few rounds of sanity tests to ensure that, the overall environment health is in good condition and scripts are executed with expected response/result. After successful sanity check, the actual test execution starts as mentioned in the test plan/schedule for required duration. Typically, it starts with the component level testing and ends with long duration SOAK tests. The most preferable way to ensure the consistency of the test results is to execute the same test at least two to three times in iterations and capture the required metrics. For executing performance tests various tools such as HP Performance Center/Load Runner, Apache JMmeter, NeoLoad , WebLoad etc. are mostly used.
Step-5 (Monitor, Analyze & Fine Tune) :
During the test execution various performance metrics are needed to be monitored thoroughly and captured for the analysis. In order to monitor the required metrics i.e. response time, errors, CPU, I/O rate, disk utilization, threads , request/response queues , data base queries etc. various monitoring methodologies/tools are used. The fundamental approach for monitoring is to use various commands in the command line interface , manual log analysis and various inbuilt tools such as AWR reports, Websphere PMI, Ops Manager etc. There are many advanced tools in the market which make the monitoring and reporting part seamless by automatic instrumenting and storing the historical data ; to name a few — New Relic, Dynatrace, BMC, CA Wily, AppDynamics, Data Dog, Splunk, etc. are used very widely. If any performance bottleneck surfaces in the performance test execution then fine tuning is required at application design/code/component/server configuration or sometimes a fine tuning may be required at the operating system level or hardware capacity stand point. There may not be very obvious performance issues but every attempt must be made to fine tune the overall ecosystem to perform optimally and stay in line with the SLA requirements. It may be required to run multiple iterations of performance test to come up with fine tuned set of parameters but the effort totally worth it most of the times.
Step-6 (Publish Test Result) :
In this phase of performance testing cycle all the findings/observations need to be summarized and presented to stake holders accurately highlighting if any associated risk with any of the components/transactions/applications etc. It also includes any recommendations that need to be implemented and/or fine tuned configuration parameters as derived from the exercise. Typically the stakeholders list would include Application Team/DevOps/Product Team/Business Team. For mission critical applications or new product launch a formal performance certification sign-off process can also be adopted for key stakeholder engagement/approval before production go live.
We hope this article will help you to formulate the performance testing process or strategy for your application.
Please stay tuned for the upcoming article on “Decoding Performance NFRs in 5 Easy Steps”.
