The easily forgotten test cases

Is the web server up? ✅

Is the app server up? ✅

Is the database up? ✅

Are the test data pre-loaded into the database? ✅

Whenever we are ready to start testing the application, everything has (and should already) been set up like magic ~

However when things happen magically, we may miss out checking a few test cases and only realise it when we do sanity testing on the production environment during a release. In today’s article, we will be looking at a few common areas that are often overlooked by development teams:

• Database/in-memory database
• Access token
• API requests
• Other miscellaneous area (i.e. no connection)

Without further-ado, let’s take a look at a few test cases of each area mentioned above!

Note: Whatever mentioned in this article is from our own experiences and they’re not an exhaustive list.

Database / In-memory database

Test case 1: Invalid credentials

Database setup is normally done once for each application where different credentials will be set for different environments. Once the setup is completed, the credentials will only be changed after x number of days. When credentials get changed, the team may forget to update manually within the system (depending on how the database is being set up) as it has been quite some time. Thus, it is always good to check for it. Also, if the application is having its first release, it’s possible that the credentials may be copied from the non-production environment. Thus, it’s important to verify that the credentials are set correctly for the production environment.

Test case 2: No table being set up

Creation of tables is done either manually or via scripts and sometimes there may be some error(s) occurring but are missed out by the team. Although such a case is rare, we don’t want to assume that it will always be successful and error-free.

Test case 3: No record available in the table

This is the highly overlooked area since everyone assumes that data will definitely be pre-loaded properly into the database tables prior to the release. Because of this assumption, development teams tend to not check/test. Moreover, test data is pre-loaded into the database most of the time which results in low chances of having no records in the database.

Test case 4: Outdated schema

The schema may get updated during the testing but it may sometimes be forgotten to deploy to the production environment, especially if your frequency of releases is short (e.g. 2–3 weeks per release); the schema doesn’t always change in each release. The application may crash if the error is not handled properly which might cause inconvenience to users using the application.

Access Token

Test case 5: Invalid access token

The access token must be generated before accessing any authenticated APIs. We often get too excited about testing the application’s functionalities once we are able to access the APIs and may miss out testing for invalid access token scenarios. Revoked token due to logout or concurrent login is disallowed in the application falls under the invalid access token scenarios as well.

Test case 6: Expired access token

Access token does have an expiration, in fact there’s two expiration types — expire due to idling and exceeding the token lifetime. If you leave your laptop idle for a short period of time, you might have seen how the application handles this case when the access token has a short expiration.

API Requests

Test case 7: Common response codes during failure

We are so focused on functional testing (where the API request will always return successful code 200) that we may have forgotten to test for other common response codes of an API request, such as:

• Bad request (400): invalid request
• Unauthorised (401): authentication failure
• Forbidden (403): authentication success but does not have permission to access the requested resource
• Not found (404): the requested resource does not exist
• Internal Server Error (500): unexpected error with the server
• Service unavailable (503): server is not ready to handle any request
• Gateway timeout (504): did not receive a timely response to complete the request

When we test the above test cases, we need to ensure that our application has a customised page to handle all the different kinds of errors. StackTrace should always be hidden from the users for a better user experience.

Others

Test case 8: Service breakage

It will cause inconvenience to the team when the database, backend server, or other downstream services are down. Hence, we should check to ensure all connections are always up. Look through your application’s architecture diagram to see if there are other external components interfacing with your application. Simulate internal/external component going down to see if the application handles it well and gracefully recovers when the component is up again.

There is a need to schedule a blackout period to inform the team when doing testing for such test cases, so that the stability of the testing environment is made known to the team. If there is no way to get a blackout period testing, you can also try either other environments (dev, UAT, staging, RC, etc) or spinning up a time limited environment and tearing it down once the testing is completed.

In case you need to have a checklist for the above test cases in this article, here you go~

Lastly, feel free to share with us anything you feel is important and will likely forget to test in the comments section below!

🧙🏼‍♀Team Merlin 💛
Application security is not any individual’s problem but a shared responsibility.