QA Best Practices

My Airtel App is a having a huge user base distributed on different devices running with varied Android and iOS versions. The app is currently doing transactions of multi crores per day and is the Top rated Lifestyle App on the Play store.

To cater such needs and the dynamic device and OEM problems a QA folk has to not only understand the product and business in detail but also the technical implementation and the different technologies used in both frontend and Backend.

1. Open STF -> Creating your very own device farm with remote debugging and log filtering.

Mobile App testers always face the problem of testing the application on different mobile devices. To solve such a problem, we have created a mobile device farm where different mobile devices are connected to a single system which is running an Open STF server. The OpenSTF provides us the capability to remotely use the devices just using a web browser. OpenSTF also enable the user to use the device over ADB to read the logs right in the Android Studio. Currently, only android devices are supported by OpenSTF however the iOS support in under development.

This setup also helps us to clear out the problem of wear and tear loss or to find out where the power cord is.

1. Devices Inventory is managed

2. Can be accessed directly from the browser

3. Input directly from mouse and keyboard

4. Display logs with real time filtering

5. Run Shell Commands

6. Debug remotely in Android Studio

7. Run the Automated Script directly on multiple devices

8. Drag and drop APK files

Directly connect the device to ADB

2. Charles / Burp Proxy -> Charles /Burp are both a middleware between the client and the server. Every request made from either your phone or browser can be seen in Charles with the information like Headers, Payload, request params and response.

Features of Charles Proxy tool.

• Network message analysis — Charles shows full messaging sources of all HTTP and similar TCP-based communications that pass via it’s proxy port.
• XML, JSON, SOAP interpretation — structured viewers that translate the raw HTTP content into a tree format for analysis
• Debugging HTTP connections from mobile devices — providing a proxy between an iOS or Android device and a remote site, to debug HTTP connections and behaviour that only occur on devices, including debugging video streaming issues, airplay issues, etc. that cannot be tested in the iOS simulator
• Bandwidth throttling — to simulate slower internet speeds by slowing down bandwidth/speed and introducing latency, for example to simulate a slower 3G connection.
• Debugging aids — such as repeating URL post requests to test server changes, adding breakpoints, or editing request variables.

3. JSON mocking for Southbound API’s -> Creation of a mock server which picks JSON object from a predefined JSON against a particular API call.

When you are integrating with a lot of third party systems it becomes a nightmare to deliver the end product. You have to depend on the availability of systems, the up time of all the servers, the response that they are giving and the bugs for them.

Features of JSON mocking.

• Return response to the Backend system based on predefined JSON.
• Mocking works based on a particular MSISDN (Mobile number). So if you don’t want anybody else to get the response you can do that.
• No need for restarting the server again and again to return the updated response.
• Any service / end point can be added on the fly.
• Easy to understand and easy to use.
• No dependency on the availability/ up time of the southbound API’s.

4. Monitoring of API and crashes -> Monitoring API health, creating dashboards and alerts for API performance, Crash Monitoring and Customer issues debugging.

When you are handling a user base this huge, API monitoring and the customer issues addressing becomes a challenge for the QA team. The expectation lies with the QA team to reproduce the issues faced and also be alert on the performance of the services used.

1. Kibana Transaction Logging -> Every API request which is originated from a user is written into our Transaction logs with various custom parameters which help us identify the type,Os,version,appVersion,lobType of user and in turn gives us the idea about the actions done by user.
2. Dashboards / API alerts -> Whenever any feature goes live, we ask the DevOps team to create dashboards on the API’s involved. QA team themselves monitor the API hits, amount of failures, probable reason involved and the patches required to fix the errors.
3. Fabric for Crash Monitoring -> We know that in this huge sea of devices and user base there will be cases where the app might crash for some users. Be it related to a specific OS version, Android Service triggering, Use case not tested, Don’t keep activities etc. Any crash which a user experiences goes to Kibana, which we monitor and get the logs for the same and try to reproduce at our end. The crashes are obfuscated due to our proguard implementation and we in turn from the Mapping file find the crash logs and then test around those code pieces.