FURPS+: Supportability tests domain

Supportability / Maintenance / Operations

// For all categories see FURPS+: Table of Contents

Supportability category has also another aliases: Maintenance, Operations, etc. Supportability is not about Customers Care Support, but it is about software operations support.

Supportability covers test cases on:

  • installation & deinstallation
  • upgrade & downgrade
  • rollout & rollback
  • configuration deployment
  • start, stop, restart, reload, status calls for the services
  • logging

— i.e. everything, that Lab and Operations Engineers deal with.

Installation and Deinstallation

Either you deploy your software manually, with automatic deployment system or you ship it in Docker containers — you need to test it. It is important, because even different versions of the same operating system (for example CentOS 6 and CentOS 7 or Windows 7 and Windows 8) have different set of system / default / pre-installed software that may not allow your software to be installed there. Or your automatic deployment system can have troubles with installation (deployment) or uninstallation (undeployment) of your software.

Installation and Deinstallation is about clean from your software operating system.

Upgrade and Downgrade

While Installation and Deinstallation is about clean from your software operating system, Upgrade and Downgrade is about operating system, that already carrying your software.

These operations often intersect with existing users activity and they are about system downtime. Depending on your system different graceful shutdown procedures must be implemented and tested. Graceful shutdown means, that no new connections will be allowed due to started Upgrade or Downgrade procedures and all ongoing connections are awaited to be terminated.

Upgrade and Downgrade is also about existing users data integrity.

Rollout and Rollback

Rollout and Rollback procedures can affect both features to be implemented and tested and the way they will be implemented and tested. This is especially true for infrastructure components.

For the complex, mature real time systems Rollout of software update may last for months: install a few servers, join them in pool, monitor for a while, update all servers in pool, monitor for a while, deploy for all users in current location, update other locations, continue monitoring during all these phases…

Rollout and Rollback are about minimization of possible negative impact. Rollback is your “last chance to get everything back” — so, don’t forget to design and test it.

Configuration deployment

Sometimes you need to change configuration.

Sometimes configuration could be reloaded on-fly, but sometimes only with a service restart. And the latter is about system downtime and that’s why it should be implemented with some kind of a graceful shutdown procedure (same as with Upgrade and Downgrade). But if configuration is reloaded on fly, then it often makes sense to test procedure with some load on the system / existing users data / connections, etc.

If you use automatic deployment system, sometimes this system can deploy configuration incorrectly — so, you need to test this as well.

Start, Stop, Restart, Reload, Status, …

Start, stop, restart — are commands for manual execution of maintenance procedures. They are used in Upgrade and Downgrade, in Configuration deployments.

Reload is about Configuration deployment.

Status helps in diagnostics of correctness of states of software and helps to debug possible issues. Status also could be used by automatic deployment systems to determine the state of maintained software.

Other commands possible: clear-cache, force-stop (bypass graceful procedures), force-restart, reload-cache, help, etc. — make sure, that software helps to maintain itself manually.

Logging

And finally, Logging. Logging helps Software Engineers to debug problems. That’s why logs must be simple and clear written, with correct error levels. Consider error level will be setup in Production — in order not to miss important log messages in DEBUG level.

Grab your logs during testing for different levels, make sure they are clear and help to debug and at the same time they are not surplus.

I hope you enjoyed reading and found this article useful. If so, please like to recommend others!

If you have anything to add, share, argue or to comment in this article — please, do it! Only in collaboration we can build something really amazing in Software Quality Assurance.

--

--

--

Only things that really matter in Software Testing

Recommended from Medium

How to Clear Cache In Laravel 8

Elasticsearch and LogZ at Boston DevOpsDays

“Developers, Developers, Developers!” Which platform truly answers that call? (Android vs. iOS)

Taking command of the Command-line…

Delegates, Events and Lambda Expressions in C# — Part 1

Avoiding stragglers during your next data center migration

Cookies and Session ID

Excel files: Loading from Object Storage — Python

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alexey Sotskov

Alexey Sotskov

I write about practical and effective techniques that help me and my colleagues in everyday software development and testing.

More from Medium

Why HALT Testing and HASS Testing is Important in PDLC

Most used basic Cypress Commands — Part One

Chinese Idiom Stories for Software Professionals: #22 A Good Neighbour worths a Thousand Sliver…

How to setup Cypress 10.x.x with Cucumber and Typescript