Non-functional-requirements stand for the operational requirements of the application which is very different from the functional work flow. For e.g. performance requirements such as processing time, processing capacity etc., security requirements such as user-id & password compliance etc. Please check our earlier post on various Performance Testing & Engineering Terminologies to get an overview of key concepts related to application performance. In this post we will briefly discuss how to define performance related NFRs.
1)User Experience :
User experience is at the center of performance testing requirement as explained in one of our articles. Mostly all the critical applications or business transactions are tied closely with SLA values. SLA is not just a contract between the business and customers but more of a guided promise by the business. For defining SLA , we need to understand the complexity and criticality of the transaction and its associated steps. E.g. suppose say, for a mobile banking application the login functionality can be tagged to SLA of 2 seconds where as the payment functionality can be of 4 seconds depending on the steps associated. Many at times the SLA can also be defined based on the competitive landscape, for e.g. If the mobile banking login functionality of your competitor is able to acheive the SLA of 2 secs then you may would like to follow it to make sure you are competitive enough with your digital offering capability. Also another approach is to break down the transactions to granular service / database / 3rd-party /api calls etc and follow a thumb rule of 200 milli second SLA.
2)Availability :
Availability is nothing but up-time, but it’s the most fundamental need to run a business. Suppose say, we own a shop and its closed most of the times then how can we expect to carry on business at the first place if we are not up and running for our customers. Application availability is defined and measured by SLOs. For e.g. Up-time requirement of 95%, 99%, 99.9%, 99.99% and so on, depending on criticality of the application. Its always advisable to start with a fairly acceptable up-time requirement and work your way up to reach an optimal state. Another aspect of availability is to keep the customers informed; so it’s a responsibility of the business owner/product manager to keep the customers well aware if any planned down time is scheduled in future for application maintenance activities etc.
3)Resiliency :
It is very interesting to accept the fact that, no matter how hard we try to come up with best design in terms of both hardware and software but every system is bound to fail on exceptional to very exceptional scenarios. In those cases its all about resiliency of the system/application that helps to recover. In order to keep the tangibility of resiliency factor, the concept of RTO or recovery time objective was introduced. RTO can vary from few weeks to few hours or even minutes depending up on the needs. Suppose say the RTO for a non-critical back-office batch job can be a week or more where as an online trading application must be recovered in few minutes or maximum up to couple of hours from the point of failure. MTTR or mean time to repair is a more fine measurement which would need extensive knowledge of past failures and best achievable time for the recovery. For e.g: if deployment of new production build results in application slowness and crashes then how quickly it can be reverted back to minimize customer impact.
4)Reliability :
In simple terms reliability means consistent expected result. Suppose say, we own a grocery shop and its open 24/7 but the items we sale are of worst quality then definitely its not a reliable shopping point for the customers. To keep the reliability factor quantifiable the quality of the service needs to be measured more often. For e.g. error rate of less than or equal to 2% of total number of transactions or 1% or 5% etc as applicable. MTTF or mean time to failure is the time between a system’s availability to un-available status which also needs extensive understanding of the application failure patterns and root cause of failures so that it can be calculated and projected more accurately.
5)Capacity :
Capacity from NFR stand point means the growth provisioning or year-on-year growth requirements in response to expected changes such as traffic pattern, new functional features, customer requirements etc . It’s the responsibility of the business stake holders to define the traffic requirements with a road map of at least 6 months to year. E.g. For an online shopping cart application the business team proposing a through put requirement of maximum 500 transactions per minute over next one year based on their market research/ survey. Capacity related NFRs are mostly translated to compute, storage and network requirements.
The best way to approach performance related NFRs is to analyse the application requirements from the above mentioned 5 points i.e. Users Experience, Availability, Resiliency, Reliability, Capacity and come up with the exact business requirement of your application. Let us know if you have any questions or suggestions on this post.
Stay tuned for the upcoming post on “3 big takeaways from Service Virtualization for seamless Performance Testing” !!
