First Taste of Automation Testing

Automation testing (AT) is often sought as a Continuous Integration, Continuous Development (CI/CD) solution to increased efficiency, effectiveness and higher quality for software products. Its application and usage, however, goes way beyond just “automating tests”. Stepping into Shopee as an AT QA has put this into perspective as I not only become part of the whole software development lifecycle (SDLC) but also partake in the development of AT at Shopee.

So what does automation tests actually entail? Let’s dive into it.

Testing Context

We first understand the context of the test. Some key questions to be answered here might include:

• What kind of tests are we looking at?
• What is the purpose of the test?
• What is needed for the tests to be carried out?
• What environment is this test carried out in?
• Which part of the whole architecture flow are we zooming in on?

These will be the foundation knowledge which we build our automated tests on and it is from these that we decide the next course of action to take such as reading code for white-box or grey box testing and deciding what is required to run our tests.

Building the Links and Flow Pathways

Next we construct the links and set up the proper flow for our own cases based on our context. Some things to take note here are the environment in which we want to run our test cases or how we want to adjust some dependency calls.

For example, a separate service using a specified environment configuration might need to be set up to achieve the testing target. This might also include microservices such as kafka and etcd in which, its set up is another can of worms that will be discussed in another future article.

Writing the Tests

As we move to writing the ATs, we follow a framework that has the general flow as follows — initialisation, setting up of data for test, assert and tear down. This general framework not only allows for clarity in our AT flows, but also increases readability and stability. The knowledge and context gathered above will also allow us to know how to set up ATs concisely and neatly, without asserting “too much/too little” or “too general/too specific”.

Automating

Finally, we move on to the “automation” component where tests and services are configured to run and build periodically or nightly. In Shopee, this is managed mainly by Jenkins infrastructure which has a friendly user interface and allows one to set up automated jobs within minutes. The versatility of Jenkins also allows for customisation of your test flow to suit your needs which might include when and where the ATs are run, specific git branch to run and more.

Once that is set-up, we can take a breather as we hands-off and let our ATs run…or can we?

Maintaining

Maintaining the ATs we wrote and built is another crucial step in the whole SDLC. ATs are supposed to signal when changes affect a feature’s functionality, which might have reasons stemming from there being a bug, change in logic or maybe a deprecated component depending on the business requirement.

Additionally, they might also be because of instability arising from the environment, services or the way ATs are written, such as improper tear down leading to dirty data and asserts which are too specific.

With the fast pace of changes in, an AT QA should always prioritize maintaining current cases before building more. Unmaintained ATs are hours, or even days, of effort down the drain. Moreover, building on something that is broken will only allow problems to snowball.

Here we looked at the general flow in generating automated tests which should not be viewed in a linear point of view but should be revisited when necessary. A few high-level thought processes as an Automation QA Engineer are also introduced which will be discussed in-depth in future articles.

~Stay tuned~