Mobile App Testing Checklist

As 95–98% of the user base is on mobile platform either Android/iOS for any product company. This makes the App Testing role very important & critical. Here, I will go through a different checklist which usually we should follow while testing any application on mobile devices.

1. Backend responses and conditions: Most applications are backend API driven. So data will be populated from the backend via REST API’s & mobile application has to display that data properly in the UI. Now, when we test mobile applications, we should always use a proxy. This will keep track of your application’s requests & responses. So if something goes wrong with an application, we can easily identify that issue is with a Client or Backend Server API’s. We can use Charles or BurpSuite proxy tools.

Few test needs to be consider while doing functional testing:

• Check success flow directly without any interception
• Intercept request, change input param’s value & verify how app behave
• Intercept request, change response
• Intercept request, change HTTP response code
• Request Timed out, block request & do not send response
• Client not able to send request
• Drop clients request
• Drop clients response
• Different error conditions with error messages
• If backend is having any insert/update/delete operation try to do SQL injection by putting some query as a parameter value
• 4XX error conditions:
• 5XX error conditions: Sometime app doesn’t handle this scenario, where app will start crashing
• Intelligent client handling: In case critical service goes down then the app should give proper UI which indicates there is an issue with service & we will get back soon.
• Response schema changed, app should not crashed. It should give proper error message on client
• In any case the client should not crash, it should be handled at client level. It has to be reported in the dashboard as an event, so it can be fixed in upcoming releases.

2. Usability: All critical features should be 1 click away from UI & it’s clearly visible

Few tips:

• Check UI component are well placed as per design
• Text/Font are in readable format
• Keypad should be driven by the input field. e.g. if its text field only Numeric keypad should open. similarly if it’s a text field then it should open an alphanumeric keypad.
• All spelling & grammar should be proper in text field & error messages.
• Language: If an app supports local language then we should check with at least a few top languages used in the platform.
• Colour should be as per design & it should not stress your eyes while using app
• Links should not be broken
• No jerk should be there while using an app
• Transition between different screen should be smooth
• If Swiping provided than check for swipe area

3. Performance: App performance is very critical, as client user base may include low end devices Which might go out of memory & app stopped responding.

Few points to be noted:

• App launch time: should be minimum as users may not like to wait for longer
• Page/screen load time
• Check performance on multiple devices, especially on low end devices
• When too many apps are open how app works
• In case of very high load on Backend servers, how does the app react? Response time is very high and it will slow down applications. Loader should be provided in such scenario
• Use some tools like firebase to get launch time & load time directly via app. It should not be a manual process, as it is error prone.

4. Security: Security testing is very critical as it might expose very critical points & might lead to some major mishap.

• Try to do SQL injections by sending input parameter as SQL commands
• Intercept a request and manipulate request and response such a way that it might create ambiguous data in DB
• Personal data should not be visible to clients. Also, it should not give personal data via any API request from Backend in response. This should be raised with the development team immediately.
• If data stored locally than it should be in encrypted format

5. Navigations: Transitions between different screens should be smooth & clear.

• To & Fro from each & every screen should be tested properly. Especially the back button provided by System & Custom back button implemented by Client.
• If there are too many rows then it should be paginated properly. It should be supported by the backend.
• Navigation from error state to previous state.
• Especially in Payment flow failure cases, landing should be appropriate. Even cancelling a transaction should prompt for confirmation.

6. Error conditions: Error message is a way to communicate with the customer. If it’s not meaningful then there is no use of such messages.

• Appropriate & meaningful error message
• Should not have common response for every error
• Text/view Colour should be appropriate to define error state.
• In case error message provide CTA than it should be proper & redirect user to proper screen/webpage

7. Feature Toggles: Nowadays most of the features are developed with AB testing framework. So it can be toggled ON/OFF from the backend or some third party tools.

• Feature comes with a feature flag, should be thoroughly tested for both flows existing & with new features. As in case of feature flag is OFF than it might create issues in existing flow because of new changes
• Data compatibility with old & new apps.
• Also, the feature flag should work with App Version number. If something goes wrong with a feature, we turn off the feature flag but when we release the app with this fix & enable feature flag this will probably start impacting users who are on older apps. So the feature flag should be supported by App version number. It’s like giving a condition where the Feature flag is ON & app Version >= new version.

8. Screen sizes: App must be tested on different sizes devices.

• are images/video stretching?
• Does it look proper
• Especially check for devices with notch like iPhone-X, OnePlus-6–7, Small devices like iPhone-5s & similarly on Android Nexus-5
• Check on tablet/iPad

9. App upgrades: Before every release app upgrade scenario should be tested, as it will impact the major user base & future growth of the app.

• App upgrade should be tested on the current version & previous available versions. (Installing previous version app & upgrading to going live app).
• Also, app upgrade scenarios should be tested on newer versions(Installing a going live app & upgrading it to the next version, to verify that it will support the next version upgrade).
• After app upgrade if the user was in a logged in state then it should be in the same state.
• In-App upgrade messages should give major new Feature releases which can keep user thinking to upgrade.
• If force updates are provided then users should not be able to skip it.
• Also, if force update provided than it should also block request from backend with version lower than Force update version

10. Onboarding flows: User onboarding journey is very important, as it guides user with some basic flows

• UX should be clear that any new user can understand workflows
• Different user have different likes & dislikes, there should be option to choose particular flow
• Introducing new feature should have proper guides & steps to understand feature
• It should be conditional, till what state we need to show onboarding flow.
• Also, there should be events added in Onboarding flow to consider impact on new users

11. User Input: Different types of input provided in workflows,

- Date and Times:

• Should provide a date time picker.
• Invalid date should give error message on client.
• If user can enter date then check for Past & future date.

- Inputs:

• String, numeric, special characters
• Greater than integer range, decimal numbers, -ve numbers, more than 2 decimal points
• Copy paste options
• Ascii/non-ascii Chars, null/empty/blank, Whitespaces
• Numeric separators, New line character

- Boundary values: min & max, +1/-1 against boundary values

12. App permissions: App must ask for permissions whenever required.

• User allow for permissions
• User deny for permissions
• Apps behaviour when changing already given permissions from settings (Granting/Revoking)
• App should ask for permissions only when user try to access that feature

13. Interruptions: How apps react when applications are interrupted?

• While performing any action, app goes in background & resumes again
• Click on the back button & app minimised/. Click on the App icon to open it again. Ideally it should resume from the same state. It should not kill & restart application.
• App killed & launched again.
• Incoming calls while doing some actions
• Push Notifications
• Random system level popup
• Random app popup

14. Push notifications: Push notifications are useful to get back users on your app. It should well timed, well framed & up-to the point

• Timing is very important (this is from a product point of view). At a high scale if timing goes wrong it will create a very high load on servers & it might go into a stale/unresponsive state.
• Push notification should be given in batches, so it will not create huge impact on server
• CTA should be proper. Users should land to the appropriate screen where it was planned to.
• User should not get multiple PN

15. App Actions: Users can do different actions intentionally, so validate it before taking it live.

• Click
• Multiple clicks on a particular button: It should be disabled.
• Dragging icons or any elements
• Hold & release after longer time
• Do action in quick succession
• Do action before getting response from previous request
• Scroll up & down while it loads all elements

16. Events: Events are more critical as it defines success/failure of any feature. For every feature it should have a set of events with a proper data point.

• Event name & property should identify different events
• Events metrics should be created with the help of some analytics tools