High Efficiency Tool Platform for Framework Migration

The eBay framework teams implemented a brand new tool platform to proceed framework level migration. It supports in-scope V3 stacks migration to Raptor/Raptor.io with high efficiency, which has been proven by the success stories of Batch, SOA, Trading/Shopping API and BES migration.

By Paul Zhang and Vivek Gadiyar

Introduction

On the eBay platform, there are four existing frameworks — V3, Raptor, Raptor.io and Node.js. V3, the oldest framework, has been part of eBay for more than fifteen years. The most important pools, like those associated with businesses like shipping, selling, and buying domains, still use the V3 framework to serve public traffic. The below diagram is a glance at the V3 framework.

As illustrated in the diagram above, there are five types of applications in V3:

● SOA is a service stack based on SOAP

● Trading and Shopping are a service stack based on Axis, mainly for customers of eBay

● BES is a messaging stack based on Database

● XSL/V4 Pres is a web stack for the presentation layer

● Batch is a batch stack for offline processing

All the applications reside in V3 environments. However, environments like Windows OS are out of support from the vendors, which means eBay must remove all the V3 footprints. There has not been much progress made because the application teams need to change code to adopt the new framework, which poses demanding challenges. It should be noted that V3 migration has already been run several times in the past.

The framework team will now cover everything in migration until production traffic is fully migrated. Customers only perform necessary validations and sign-off the pool for each environment, like staging and production. This approach is called forklifting.

Our first challenge was to make the migrated code work as it had before, and here’s how we did it.

Requirement

The below diagram briefly describes how the architecture looked after migrating to Raptor/Raptor.io.

First, we migrated to new implementations, which involved more than just changing terms. It required a holistic review of three frameworks (V3, Raptor, Raptor.io) and four stacks (SOA, Trading/Shopping, BES, Batch) which involves more than 500 application changes.

The main three tasks:

1. SOA/Trading/Shopping are migrated to the Raptor.io framework
2. Batch/BES are migrated to the Raptor framework
3. XSL/V4 Pres are migrated to Node.js by customers

Second, we addressed issues in common areas like kernel that might result in different errors by the stack. Since each stack has dedicated owners, this posed three challenges:

1. The knowledge could be shared with crossing owners.
2. The dedicated problem couldn’t be solved with a unified approach.
3. The dedicated owners couldn’t resolve the framework difference with a unified solution

To build a unified migration tool platform, the framework team needed to have a concrete and unified solution to address the aforementioned issues.

Approach and Design

To build a migration tool, we first needed to identify the common patterns and the many differences between V3 and Raptor/Raptor.io. For each individual application’s migration, we considered two perspectives: resources and runtime.

Resources

● Code structure

● App metadata, like pom.xml, raptor_app.xml, etc.

● App configuration

● Dependent stuff changes (e.g. IBM JDK 1.6 for V3, OpenJDK 1.7 for Raptor, and OpenJDK 1.8 for Raptor.io. Different JDKs may have different behaviors.)

Runtime

● Stack feature gaps (e.g. For Ratelimiter, it is a local call in V3, but a remote call in Raptor.io.)

● Stack kernel backward compatibility (e.g. V3 batch is a home-made batch core, while Raptor batch is based on Spring batch)

● Framework kernel library backward compatibility (e.g. Raptor.IO rewrites the library to components and some classes’ signatures are changed)

● Third-party library backward compatibility (e.g. Jersey version is 1.0 in V3 and Jersey version is 2.0 in Raptor.IO)

● Initialization mechanism difference (e.g. V3 is a home-made initialization mechanism but Raptor IO is Spring boot initialization mechanism)

● Instance Lifecycle (e.g. Instance construction time point is different between the v3 container and Spring Boot container of Raptor.io)

● Unexpected exceptions (e.g. The same class from different libraries has different implementations)

After we had a detailed breakdown, we categorized them into three levels:

1. Framework level problems

● Code structure

● App metadata

● App configuration

● Framework kernel library backward compatibility

● Third-party library backward compatibility

● Initialization

● Instance lifecycle

● Dependent stuff changes

2. Stack level problems

● Stack feature gaps

● Stack library backward compatibility

3. People involved problems

● Unexpected exception

Based on the above analysis, we created this diagram for our migration tool platform.

There are two platform converters — one is for Raptor, and the other is for Raptor.io. They all offer the lower level framework adoption logic and for each stack there is a plugin to handle the difference. The platform converters cover the framework difference, the stack part focuses on the stack difference between V3 and Raptor/Raptor.io. Moreover, most problems on the top of the diagram are already resolved by the layer below. The migration team only faces the common problems. The migration could be preceded by few people from the below benefits:

● Build up a shared knowledge base easily.

● There is no difference to handle the migration problem regarding different stacks.

Implementation Case Study

Platform Converters

There are eight parts in the platform converter. We took the code structure as an example.

Code Structure

V3 code structure was a composite of applications. One application possibly exists in many deliverables, and one deliverable may have many applications. Below is a demonstration:

Therefore, the implementation needed to satisfy the following:

1. Combination requirement: For some Trading/Shopping deliverables, it required all applications of one deliverable to a Raptor/Raptor.io application. We needed to modify our tool strategy to accommodate the new case.
2. Separation requirement: Some SOA deliverables had strong requirements to split the V3 deliverable into different applications, such as soa1, soacs1. They were not split into individual ones, but rather a smaller group.
3. Mixed application requirement: Some trading API /BES deliverables contain SOA applications. The tool also needs to cover SOA applications.

To meet all the requirements, we needed to refine our solution. The applications of the deliverable were separated into two types. For the applications that exist only in one deliverable, they were called non-shared applications. For the applications that were shared for many deliverables, they were called shared applications.

The rules for migration:

● All the non-shared services are migrated as source code applications.

● All the shared services are the maven artifacts built by the V3 train release.

● A new entry project is created as a Raptor.io/Raptor running project. The non-shared project and shared application are the dependencies of the new project.

● The tool will support the parameters like deliverable -services -sharedservices. The maximum number would be the whole deliverable applications, the minimum number would just one application. Considering the mixing cases, like one deliverable contains trading application and SOA application, the parameters would be like -services -sharedservices -tradingcommands — shared tradingcommands.

The migration tool platform had maximum flexibility from the design level once we broke into the below code structures:

Stack Plugins

We implemented five stack plugins in this tool platform — below is an example for Batch Plugin.

Adapter Library

The challenge for the batch stack plugin was that V3 batch is based on a home-made batch framework, while the target migration Raptor batch is based on Spring batch — needless to say, they are different kernels. For example, the V3 batch task class is a Java class called Task, while the Raptor batch task class is a Java interface called Tasklet. If we couldn’t find a way to accommodate the kernel differences, the application code would change considerably, since the kernel APIs are vastly different.

The basic rule for migration is to maintain the customer’s business logic code. Batch stack plugin must be able to overcome the difference and make the existing code work well.

From the design pattern of view, it should be the adapter pattern, so that the application’s business code won’t change even if the core logic is changed. The migration team needed to build an additional adapter library. Therefore, the whole V3 batch core classes were considered as APIs, and class signatures were kept as before, only using the Raptor batch way to implement it again. It required the migration team to have thorough knowledge of the classes, lifecycle and initialization difference in V3 and Raptor batch core.

Feature Gaps

Each stack of the framework has its features. The migration team needed to do the following actions:

● For features that behave differently, we used the same behavior with V3, but wrote in the Raptor batch way.

● In the case where some features were supported in V3, but retired in Raptor, like DSF and ESF, different strategies were applied.

• In batch, all the run-time logic associated with DSF/ESF were email functions. Since the messaging team already offers pair functions, the migration team offered a detailed solution to help migrate the function.
• During the initialization, some ESF and DSF initialization modules were called by transitive dependency, but they did not interfere with the business logic. The team made configuration or code changes to bypass this kind of module.

We introduced the main design concepts for the two cases above. With this tool platform, code migration was not a blocking point for the V3 migration team. Any application can be migrated within two minutes, and the migrated application will test on staging about one day after the migration team fixes the common problems.

Conclusion

The framework level migration is a brand new methodology which benefits the V3 migration but also provides a reference for any framework migration. During the migration, we had learnings on both the horizontal and vertical view. From the horizontal view, we have knowledge for five stacks between V3 and Raptor/Raptor.io, and from the vertical view, we have encapsulated the complexity that brings the little effort to make an application ready for testing and rollout quickly. During the process, the migration team not only helped the domain team to remove the V3 footprint but also built a strong knowledge base for each stack. The tool platform is working as expected because the team can go deep to further investigate the unprecedented challenges and break them down into actionable items to tackle step-by-step.

An easy life of domain teams is always the goal of the framework team.

Originally published at https://tech.ebayinc.com on September 16, 2020.