What do you really need within your solution: ESB or Microservices?
Digital transformation is becoming essential for any organizations which expects to enhance their business values, explore new opportunities and performing ahead of their competitors in the market. There are several different integration methodologies, adhering to different architecture patterns. It has been more than a decade when the Service Oriented Architecture (SOA) emerged into the industry. Enterprise Service Bus (ESB), backed by SOA, became the most popular middleware pattern to solve most of the problems had with traditional monolithic applications. However ESB could not hold its position as the leading middleware implementation, when cloud based technologies, automation and the infrastructure has been evolved with more and more features to build more scalable, reliable applications. Microservices architecture has become the next strategic move for the integrations which uses the full benefits of cloud infrastructure to build resilient, auto healing, scalable solutions.
If an organization is about to start digital transformation, one of the first questions would be the selection of implementation and architecture : Would it be ESB based, Microservices based, hybrid solution or another totally different methodology. When there is a bunch of options available for your new integration solution, it is worth seeing what are the reasons for you to choose a specific option against all others. There are benefits as well as drawbacks in each option discussed above. Lets see what are the key differences in ESB and Microservices which will help anyone to take a decision on their future solutions.
Enterprise Service Bus (ESB)
ESB is a middleware architecture based on SOA. It is based on service providers, service consumers and event flows. Service providers generate events and publish to the event bus which is known as Enterprise Service Bus. Service consumers consume events they are interested in. ESB supports both synchronous and asynchronous events flows.
ESB comes with a number of different features required while messages are passing through it. It includes message transformation, protocol conversion, service orchestration and payload enrichment. These features are readily available in all ESB implementations from different vendors. It helps the solution to be integrated with different types of systems within the enterprise.
When an integration solution is built, ESB alone would not fulfill all the middleware requirements within the scope of requirements. There are few other middleware platform capabilities will be required;
- API Management — to expose internal resources to external world with proper security and QoS
- Messaging Queues — to build asynchronous communication with external systems.
- Business Processes — to engage humans or equipment within message flows in certain validations.
- Identity and Access Management — to manage access within the solution.
- Monitoring and Alerting — to identify run-time issues and act fast.
Following diagram shows how different systems can be connected via ESB.
Advantages of ESB
ESB has several benefits in terms of service integration.
Technology independence : ESB supports different types of transport protocols such as HTTP/S, AMQP, MQTT which makes the technology of the external service is not a concern. Any service which is built on any technology can be integrated as long as it can communicate using one of the above protocols supported by ESB.
Service re-usability : Once the services are exposed through ESB as virtual services, those can be used by any other service regardless of the limitation of networks and technologies. New integrations will require minimum effort to be added to the solution.
Centralized governance and monitoring : ESB has the traceability of all services and therefore it can be the central point to govern service usages and monitor statistics. If the services are connected through a point to point connectivity, then the monitoring should be happened at each and every connectivity.
Maintainability : There is no tight coupling among the end services and therefore those can be independently updated without affecting other services. Also different types of controlling can be applied at ESB layer rather than updating actual services.
Fault Tolerance : Service failures and recovery procedures can be handled at ESB and end services will not be affected. E.g if a service failed, ESB can route messages to its fail-over service endpoint until the original service comes alive.
Deployment is not complex : ESB has a simple deployment method which will contain all service routing and orchestration capabilities built in. Different vendors may have different types of deployment patterns, but eventually, it will not be too complex to be handled.
Disadvantages of ESB
Added delay in round-trip : ESB layer will add additional step in the service flow and add a slight delay to the response time. Therefore the performance might not be the same as accessing the individual service directly through a point to point connectivity. However, it is a compromise between the minor delay added against many other advantages given by ESB.
Connectivity requirements : All services require to have connectivity to the ESB to facilitate service invocation and this will incur additional cost to setup VPNs and other infrastructure.
Single point of failure : Having all services routed through a single place, it will have a risk of single point of failure. This can be avoided by having proper failover (client or back-end failover) and auto healing capabilities of latest hosting platforms. Many ESB vendors support sophisticated high availability deployment patterns to avoid these failures.
Complexity in scalability : Even only a single service is required to be scaled up, there is no such way to increase the capacity of that service independently. Instead, it requires to scale up the whole ESB with all other services as well. It will be a waste of resources and the cost.
Microservices
Microservices Architecture (MSA) is more popular today compared to Service Oriented Architecture. With many fascinating features provided by the latest cloud technologies and platforms, it is fair to say that Microservices deployed in highly scalable, reliable cloud platforms give more value then ESB in SOA. Each microservice has its own business functionality. Therefore, compared to traditional ESB, Microservices have more fine grained control and higher scalability.
Following characteristics are available in a typical microservice.
- It has a unique scope of business functionality. If you need to get consolidated functionality, then the option is to integrate few microservices and build a composite microservice.
- Each microservice has its own database. Centralized databases are discouraged and will not provide the value of fine grained control.
- Microservices are very loosely coupled. No direct communication between each other in core microservices.
- Each microservice will have its own runtime, mostly running on its own container.
- It has very low startup time which provides high availability in case of failure and also the capability of scaling.
- Microservice can be independently developed, deployed and scaled up without effecting whole system.
- Microservices deployment would be complex with lot of communication around it. Therefore it is mandatory to have CI/CD in place.
- The success of microservices deployment depends on communication between microservices, and therefore need highly stable infrastructure.
Following figure shows how microservices are placed within a solution in different layers for the convenience of management and communication.
Advantages of Microservices
Highly Scalable : Self contained services can be scaled independently of other services based on specific requirement of the business functionality.
Technology independence : Microservices can be built using different technologies and deployed in different platforms as long as communication can be established between each service. Therefore the development would be more flexible with the skills available in the organization. Most importantly, implementation technology can be selected based on the performance requirements. For example, high performance real-time applications can be built using a lightweight, performance friendly technology while bulk data processing services can be built using a technology which has caching and data streaming capabilities.
Service Resilience : Microservices are designed based on the principle that errors are raised at the earliest point which helps to trigger failovers and recover within a short period of time. It will avoid the whole echo system failure as a result of the fault of a single service. Architecture patterns like circuit breaker and timeout will help to achieve this requirement.
Supports cross team agile development : Each microservice can be developed and deployed by different teams and therefore the business does not require to wait until all teams are complete their work to see the result in production.
Quicker go-to-market : Microservices development will involve comprehensive CI/CD implementation and therefore releases can be planned and executed in parallel, without any dependency on each other. It will result in faster go-to-market strategy for the business.
Disadvantages of Microservices
Infrastructure requirements : Microservices heavily use features of cloud based container services to get the maximum advantage of scalability, reliability and fault tolerance. Therefore it requires specific infrastructure with comprehensive automation capabilities to achieve maximum benefits.
Complex communication : The success of microservices deployment is completely depends on the network communication. Since there are a lot of services to cater complex business functionalities, network traffic will be higher and require reliable network infrastructure.
Supporting transactions among services is not easy : Independent services do unique set of functionalities and therefore it is difficult to handle scenarios like transactions with standard microservices. It requires additional design and development effort to implement such scenarios effectively, while maintaining other microservice features unchanged.
Difficult to migrate from monolithic : If the organization already has legacy applications built on top of monolithic architecture, it would require an extensive effort to convert it to micro services and moved into a different infrastructure.
ESB or Microservices : Which is the best?
When you select the suitable solution for the organization, following factors need to be assessed.
- Types of services to be integrated
- Communication protocols required
- Performance numbers
- Transaction requirements
- Security restrictions
- Inter-dependencies of services
- Business timelines
- Target expenses for the infrastructure
- Skill set of the team
It is obvious that microservices give added advantages with the rich set of characteristics it has, supported by the latest cloud deployment platforms. However, there are few advantages and drawbacks in both implementations as mentioned previously. There is no unique answer to say that one implementation is the best. Based on your requirements in your organization, one of these solutions will be better than the other. The requirements should be carefully analyzed along with the growth of the organization’s Digital Transformation Strategy in next 5–10 years, before taking a decision on the option to be used. Even a hybrid approach would also work as long as the organization business goals can be achieved through the solution.
References:
https://developers.redhat.com/blog/2017/08/02/organizing-microservices-modern-integration/
https://cloudacademy.com/blog/microservices-architecture-challenge-advantage-drawback/
