Mule 3 to Mule 4 Test Driven Migration

When performing a Mule 3 to Mule 4 migration, we want to make sure that the new Mule 4 APIs return exactly the same results as the previous Mule 3 APIs. We also want the performance to improve or, at least, remain the same. So obviously there is a big need for tests and documentation after the migration is over. Also, there is clearly a big room for test automation because manually testing hundreds of flows is not efficient and is error prone. In this article we will be looking at how we can apply Test Driven Development (TDD) principles to a Mule 3 to Mule 4 migration project and what are the benefits of using this approach.

The Need for Automation

Imagine an organization with 10 APIs that need to be migrated from Mule 3 to Mule 4. Each API having an average of 10 endpoints and each endpoint with an average of 5 possible scenarios, i.e. data combinations that produce different flows. In this case, the test team would need to run five hundred requests on Mule 3 and five hundred requests on Mule 4, grab the results, compare them one-by-one and document the output. They would also need to keep track of response times to evaluate the performance. Considering that each manual test takes an average of 10 minutes, it would take 166 hours to complete the whole migration test! Moreover, relying on manual tests would never give you enough confidence that all results are 100% accurate and it would be such a huge task force that it would probably be done once or twice in a lifetime. So it makes a lot of sense to automate testing and document generation during a Mule 3 to Mule 4 migration.

Test Driven Development Principles

Frequently, testing only starts after the migration is done. However, it is common sense that it is an essential task that should never be overlooked. So, if we know that we will be doing this effort anyway, we might as well consider the benefits of creating the automated tests before the actual migration is implemented, using a Test First approach which is one of TDD principles.

In fact, many TDD principles seem interesting tor this kind of project:

1. Red-Green-Refactor Cycle: This is the process of writing a test (Red), writing the minimal code required to pass the test (Green), and then refactoring the code (Refactor).
2. Write the Simplest Code to Pass the Test: Developers aim to write the simplest code that fulfills the requirements of the test. This means avoiding unnecessary complexity.
3. Run the Entire Test Suite: After writing code, the developer runs the entire suite of tests to ensure that the new code doesn’t break any existing functionality.
4. Write Clear and Specific Tests: Tests should be clear, specific, and focused on testing one piece of functionality. They should be easy to understand and maintain.
5. Tests Should Be Fast: Tests should run quickly so that developers can run them frequently during the development process.
6. Tests Are Documentation: Tests serve as living documentation for the codebase. They provide examples of how the code should be used and help to prevent regressions.

The first TDD principle, Red-Green-Refactor Cycle, tells us to create the tests before the code. In a Mule 3 to Mule 4 migration project this means creating the test flows before implementing the Mule 4 APIs. Basically, this means creating a Mule application that tests all the existing Mule 3 endpoints and compares them with their yet to be implemented Mule 4 counterparts. These Mule 4 APIs can be created at this stage as just placeholder endpoints without actual implementation or even using the MMA tool. This way, the first time the test runs it would get 0% passed. So you will know that your migration is 0% completed and, as developers progress in the migration work, you will easily generate a status at any time and know how far you are from reaching to the end of the migration.

Creating a Migration Test Application

To accomplish this we can create a simple Mule application that orchestrates the automated tests for all APIs, endpoints and scenarios and generates a migration status report at the end.

We can start by running a simple readiness check so we can be certain that all dependencies, such as databases, message brokers, downstream APIs, etc. are ready and accessible for both Mule 3 and Mule 4. If the systems are ready then the application starts testing each API.

Migration test main flow

For each endpoint, the application can load test data from a CSV file and send, in parallel, the same request to the existing Mule 3 endpoint and to the new Mule 4 endpoint, save each result in individual files and finally consolidate both results.

Testing a GET endpoint

For POST, PATCH and DELETE operations there will be some extra work to fully automate the tests. This can be accomplished, for example, by querying the backend after the request took place and checking if the operation produced the desired result such as insering, updating or deleting a record.

Testing a POST endpoint

Generating Migration Reports

For each environment, Mule 3 and Mule 4, the application generates CSV reports with all output data: Status code, response and elapsed time for each API/endpoint/scenario and generates individual reports.

Mule 3 test report

Mule 4 test report

At the end of the process, the application compares each result and generates a high level status report which will give you a detailed and accurate view of how much work is still needed to complete the migration. It will also show you how accurate the new endpoints are in getting the same results as the old ones. Finally it will also provide you Mule 4 performance results compared to Mule 3 APs.

Migration status report

Benefits of Test Driven Migration

With this approach the organizations have a reliable, efficient and easy way to:

• Test if the new Mule 4 APIs are getting exactly the same results as the previous Mule 3 APIs.
• Test if the performance of the new Mule 4 APIs have improved, remained the same or deteriorated after the migration.
• Know exactly how far the migration is from completion just by referring to the migration status report.
• Regression test. After each specific endpoint migration, the developer can run the entire suite of tests to ensure that the new code didn’t break the endpoints previously migrated.
• Provide Documentation. Tests serve as living documentation for the migration. They provide a report of the migration status.
• Support the post-migration stage by checking if new bugs or defects were caused by the migration or not. Also providing evidence of Mule 3 results if users question the Mule 4 results.

Test Driven Migration major challenges

A few challenges can be faced during this Test Driven Migration approach, however, most of them are problems that would need to be dealt with at some point during the migration regardless of which method is used.

• Set up, sync and isolate SIT (System Integration Testing) backend environments. To be able to validate these endpoints, Mule 3 and Mule 4 environments need perfectly synced and isolated backend systems.
• Non-Idempotent operations such as POST, PATCH and DELETE methods require more attention and effort to develop your test flows.
• Asynchronous flows with heavy use of queues and/or topics can also add complexity to design your test flows.
• Convince developers to follow the “write a test before coding” approach. Developers tend to do testing at the end of the cycle and might need a little push to break this habit.
• Sometimes the test planning starts when the migration has already begun. Nevertheless, you should still get most of the benefits of creating the automated tests for a migration project that is already in progress.