Stop forcing reusable components on the Integration Layer.
Next-gen Integration layer architecture.
One of the major design drawbacks of the traditional ESBs is their close tight coupling with the provider systems and letting multiple consumer systems reuse the same adapters (services) just because they initially seemed to be consuming similar data (payload).
If we look closely, the tight coupling to the provider systems is something that was forced upon ESB by the provider systems. Most backend core systems used to communicate with SOAP over XML which meant any change in the services at their end has to have a similar change at the ESB system to ensure the communication don’t break. The paradigm shift towards granular APIs at the backend provider systems meant they no more force tight coupling onto any systems they communicate with.
Offering one API for multiple consumers is a tactical design approach that integration architects and the Enterprise application owners make together. There’s absolutely no problem with that. But knowing when to split these consumers into consuming different APIs and how to sneak in a few placeholders to help the same is up to the integration architects.
I worked for an Indian insurance giant helping their digital transformation journey. As part of it, they designed a new web-portal and mobile app exclusively for their agents. At the outset, the app and the portal seemed to be a mirror image of one another. This integration layer was the backend for their front end channels (agent web-portal and agent mobile app) inturn the integration layer orchestrated multiple core vertical systems like PAS (Policy Administration System), CRM (Customer Relationship Management), LMS (Lead Management System) etc. For phase 1, we went in with the tactical approach of both the mobile app and web-portal consuming the same integration layer APIs (same endpoints).
After phase 1, the business wanted certain mobile app specific enhancements as part of phase 2. This is where we took a decision of splitting the IL APIs such that each consumer (mobile app and web-portal) consumes a different API at the integration layer. And though at first, the application owners at the insurance company were circumspect of having two APIs doing a fairly similar job, As phase 2 progressed the Agile requirements brought in more and more complex enhancements at the mobile app within strict timelines. JFM (JanFebMar) is an extremely crucial time for the Indian insurance industry and the business wanted the rollout to happen immediately. Too many enhancements meant complete regression testing which also meant extra days. Since the web-portals are consuming the exact same APIs they did before and since those APIs (codebase) are not changed (meant only the mobile app needs to be tested) resulting in cutting back 50% of regression testing efforts and timelines.
Halfway through phase 2 implementation, the business team wanted a completely different set of features in web-portal for phase 3. /renewaldetails which was previously supporting both portal and app couldn’t have expedited these implementations. Whereas /agentwebportal/renewaldetails and /agentmobileapp/renewaldetails have effortlessly supported the timelines thereby making Business and IT teams happy.
Let’s take a step back to think what’s wrong with having the same API supporting both the channels.
- While one Sprint (agile Dev cycle typically 3 weeks) is running on an API for a channel, other channels have to wait to implement their changes if their cycle starts mid-sprint of the ongoing one.
- Not all channels expect the same payload. Forget about transformations, Sometimes there are derivation logics. If the same API has to be consumed, we force the consuming systems to derive/transform what they want by themselves which is horrible because that’s the very reason for the existence of an integration layer.
To put it simply, the biggest problems of traditional ESBs are more because they were rigid to changes and that’s mainly because they were designed to reuse their components across multiple consumers. In today’s world where enterprises are racing to capture markets with new ideas, the IT should enable them rather than delaying the go-to-market dates with complicated rigid systems.
The idea is only as good as the time of its execution.
