“Testing, Testing, 1-2-3": Shift Left Methodology and Bug Bash

By Shyam Krishnamurthy and Naveen S.R

In today’s world, the technology team is tasked with delivering complex and challenging initiatives in a time-bound manner and this needs skilled resources who work together as a team. At AirAsia our tribes follow agile development methodology with the intent to handle changing requirements, getting early feedback, and avoiding late surprises from our stakeholders. Our focus is always on customer experience and the quality of our applications.

In this blog post, we will discuss the Shift Left methodology at AirAsia and also specifically about Bug Bash as a Quality Driver.

Shift Left Methodology

Shift Left Methodology advocates identifying issues as early as possible in the software development lifecycle. By identifying issues early in the cycle we can save costs, launch software quickly with better quality. We use extensive Unit testing, functional testing, and API contract testing in our CI / CD pipeline for detecting bugs early in the development cycle. QA Regression testing and Bug Bash completes our testing arsenal and helps us make good quality software for our customers.

Unit Testing

To ensure higher code quality, all our developers write Unit Test cases for their software. Every software unit built needs to have Unit Test cases so that we can test it for its expected behavior. Our goal is to achieve a Unit Test coverage that is greater than 95%. All the Unit Tests are linked to our CI / CD pipeline and are run with every build. This helps us to detect any code breaks early before the code is deployed to higher environments. We have unit tests written in Junit and Jest.

Integration Tests

Integration tests ensure that the software works together with all its dependent systems. These tests expose the issues that arise when disparate software systems are integrated. For example, integrating the online flight booking system with the order management system and ensuring transactions are successful.

API Contract Testing

Contract testing is essential to ensure two dependent systems can continue to communicate as expected after changes. Example: We have an API platform that serves web, mobile, customer service bots, etc. Any changes that happen should ensure backward compatibility of existing consumers seamlessly and ensure schema validation. Contract Testing is written with Pact and Karate framework.

System Tests

System tests help us test complete integrated product experience. We run load tests based on expected throughput and identify potential scaling bottlenecks, memory leaks, etc. Besides this we also have vulnerability tests plumbed into the CICD pipeline to facilitate automated push of code across environments.

UI Tests

Our mission has always been to ensure the best experience for our customers and therefore we strive to produce high-quality software. We believe that quality is a shared responsibility owned by developers, QA, and the Product team. To bring in a culture of ownership and accountability, we have tasked our developers to write Functional Tests. These tests are written from a user perspective i.e. input is provided and the output is validated for happy path and exception scenarios. UI tests are written with Selenium.

QA Regression Testing (Automated + Manual)

Our QAs are developers who wear the hat of the QA as well as specialized QAs who are domain experts. Before the Regression Testing, we undertake a strong release management process that comprises a thorough check of Functional Reviews, Lighthouse Scores, Accessibility Checks, SEO compliance, Load Testing, Review of the Quality Boards.

We identify the critical path test cases for every Line of Business at AirAsia. Our regression suite will ensure that every software change goes through the P0, P1, and P2 testing. A major % of our regression suite is automated and we invoke suites of suites across LOBs. So we rely on a combination of automated and manual regression testing to certify builds. Our goal is to keep the manual testing to a minimum to ensure faster certification of builds.

Bug Bash

A community testing session organized by the development team inviting developers, Engineering Managers, Product Managers, QA, and Commercial teams so that they can play with the product and discover bugs. This exercise intends to enhance quality by discovering issues internally before we launch it for our customer use.

Each Bug Bash session has a Captain who sets the stage for the testing. The testing is divided into several tracks and each track has a leader who will help the community of testers if there are any challenges in the testing environment or if they need clarification during the testing etc. The session starts with a video/demo showing the happy path feature that needs to be tested. The community of testers can begin their testing as soon as they are ready and any issue found during the testing will be logged as a bug in our tracking system for further action.

About the Author

Shyam Krishnamurthy is the country head of AirAsia Technology Center based out of Bengaluru comprising of about 80+ amazing talented engineers. He heads the travel vertical with a Line of business responsible for Flights, SNAP, Hotel, Post Booking and Mobile Storefront experience.