Spring boot migration: Learning from our pursuit of efficacy
At Chegg, one of the core engineering principles is to create generic solutions. This allows for different engineering teams and developers to reuse these solutions without much tinkering. One way to do this is to build tools or services while keeping in mind solutions that favor long-term functionality over short-term-focused and single-purpose software.
In that vein, Chegg recently started to migrate all our back-end java projects to spring boot. We still have a long way to go if we want to reach a point in which all services are empowered by spring boot. This migration allows us to build a long-term generic back-end platform, which leverages the capability of open source and java’s new features. This effort involved bringing together teams across the organization to migrate the new technology seamlessly. This post shares some of the lessons we learned during one of our spring boot migration projects.
Background
Long story short, Chegg has its home-grown platform to empower our back-end services. Several years back, when we built a platform for Chegg’s back-end services, there were not many good options which could fulfill our needs. Most of the services were running with the legacy platform. This platform served us well and provided us with some generic solutions such as:
- Configuration Management
- The standard for REST Mapping and Validation
- Exception Handling, Logging and shared defaults.
- Client library generation
And some other functionalities.
Several years back, when we started migrating to docker, we found out that the platform has a lot of restrictions which were becoming a hindrance to the full capability of container technology. Also, we required additional resources to improve and revamp the whole platform continuously and needed to train new engineers to ramp up this platform. It would take some time for engineers to get comfortable with the platform before they could deploy their code, preventing new hires from focusing on developing business solutions.
After long brainstorming sessions and various discussions among different teams, we decided that spring boot was the solution for our back-end services.
Learning
Invest in tools
As we always strive to find generic solutions at Chegg, we decided to build the tool with the following goals:
- Smooth/Seamless migration to spring boot with minimal code changes.
- Converting the code from Chegg-specific platform standard to java standard.
- Artifact usage and integration.
When we started building the tools, we aimed to make it generic enough to convert primary legacy platform standard to java standard. However, once the tool was ready, it wasn’t perfect. However, it does a decent enough job to accelerate the migration. We already have used the tool for multiple project migration to spring boot until now, and it makes the whole migration process a little less painful and scary.
Testing as a savior
This migration is not adding new functionality to the customer and, instead, it’s using new technology (in this case spring boot) to empower existing functionality. One of the challenges these kinds of migration pose is breaking any existing functionality.
At Chegg, we use different tools to keep track of our code coverage and each team tries to make sure that they have near to 100% code coverage. This practice turned out to be useful when we started migrating projects. Before migrating, you want to make sure that all the tests (including Unit, Integration and E2E tests) are relevant to the code and passing. One of the main reasons for successful migration is that we have good code coverage and End-to-End tests.
Spring boot provides several useful annotations to create tests cases. We have used spring-boot-starter-test as a dependency in our current project which provides Junit4, Spring Test, AssertJ, Hamcrest, Mockito, JSONassert, and JsonPath for free. For our project, spring-boot-starter-test gives us sufficient functionality so we don’t need to add any external dependencies.
Spring provides @SpringBootTest annotation to run the test. You also need to add @RunWith(SpringRunner.class), which is necessary to run the test in Junit4 but you don’t need to add this on Junit5.
Spring boot has excellent documentation and tools which makes it easy to write tests.
Look out for dependencies
Resolving dependencies is one of the most painful parts of the migration. We faced a lot of versioning and library conflict. You have to invest time to debug and research to resolve these issues.
One of the ways you can approach this problem is to divide and conquer:
- First, copy all the dependencies in your newly built spring project without modifying the version.
- Bump the version of the specific library and run all the tests.
- Resolve issues if you can with the new version.
- Revert to the previous version if you can’t resolve the issues.
- Repeat the process of step2 — step4.
Refactoring
One of the things we do while migrating is refactoring our legacy code to new java techniques, like lambda and stream. We were able to refactor a lot of our classes and methods and make them more concise and straightforward.
One of the benefits migrations give you is you can refactor your code freely. Learning here means taking a hard look into your existing classes and methods, trying to improve them by introducing a new java technique or, in general, making it more straightforward.
Be Cautious: Database and Caching
Spring boot makes it so easy to integrate different kinds of databases. We entirely rely on spring boot database libraries to access our different databases. So far, we haven’t encountered many issues.
We had removed much of the custom code that was used to facilitate access to the persistence layer. However, replacing that code with spring boot was easy since spring boot made the process to access persistence very painless. While doing all these code changes, we were very diligent about fixing the tests and writing new ones right away instead of doing it later. So, it’s essential to keep fixing and adding new tests based on your code changes.
As far as caching goes, we rely on our caching solution due to some specific scenarios. However, spring boot does provide you with excellent tools to integrate and access caching without much code. Based on our experience, this varies from project to project. Find out what best suits your project’s needs.
In most of our project, we went with spring boot persistence and caching libraries. However, it’s also true that we rely on our legacy code to access DB and cache since replacing them was not justifiable considering the effort.
Configuration
Spring boot is a configuration driven framework which gives you much configuration without any efforts. Understanding those configurations and how to leverage them is essential. We have configuration, not only related to code but, also related to containers, monitoring and logging.
Fortunately, we have in-house sprint gurus who have helped us during the process to define and understand different configurations. We did write custom code and scripts to facilitate defining rules for our monitoring and logging system. However, we observed that most of our configuration code had been reduced significantly due to spring boot various tools.
Adding a new configuration and removing old ones need some strategic thinking. You want to make sure that it doesn’t become maintenance overhead.
One of the good examples could be a health check endpoint. Spring boot provides its health check endpoint and we wanted to use that. To prevent any downturn while we switch from the old health check to the new one, we temporarily added the old health check endpoint. Once service has deployed, we remove this temporary endpoint and switch to spring boot health check. We wanted to make sure that we are not maintaining extra code in our spring boot application.
Some Other Valuable lessons:
Besides technical challenges, we also learned an excellent lesson as a team and individual engineers.
Understand limitation
There is always a limitation, how far you can go to make a perfect solution. We bump into the dilemma of completely switching everything into spring. However, we have some parts which still have legacy system references, like our endpoint, that still hasn’t changed; they still refer to the legacy system names. However, we understand that, sometimes, it is better to take the iterative approach and continuously improve other projects.
Collaboration is key
We had a lot of discussion and sharing of knowledge across the team which helps users learn from each other. Every team, which migrated their services from legacy to spring boot, shared their learning. And, so did we. Sharing helps not to repeat the same mistake that another team makes.
One of the things which we did from the start was documenting all the learning and steps. We are also excellent at making sure that our documentation is up-to-date and has relevant information. This way, the next team can learn from them and not repeat some mistakes.
In Conclusion, we see these as valuable lessons and we continue to improve some of our upcoming, new migrating projects. Considering the trade-off and ROI, we think these lessons taught us a valuable lesson in the migration of our back-end technology.
