Flying as a Flock: Helidon and Coherence Integration in Oracle Banking Cloud Services
We knew, when we decided in early 2020 to develop integration between Helidon and Coherence, that the combination would be powerful and productive for architects and developers building high-scale microservice architectures accessing enterprise data. The marriage was natural, with Helidon providing a state-of-the-art Java microservices framework, and Coherence providing proven reliable scalable data management and grid computing capabilities for Java and other languages.
Oracle’s Financial Services Global Industry Unit (FSGIU) recognized the pair’s potential, within a domain-driven design architectural style, and has now realized the benefits in its Oracle Banking Cloud Services offering launched last year. Built with Helidon and Coherence, the Oracle Banking Accounts Cloud Service (OBACS) within that offering has achieved breakthrough scalability within the corporate banking industry for demand deposit account (DDA) solutions. Time to benefit was also unprecedented, with scalability testing showing linearity about a half-year after development began — in a very complex problem domain requiring the utmost data integrity, with highly concurrent banking transactions affecting the balances of multiple accounts simultaneously.
We’ll go into detail about the development productivity and architectural performance enabled by Helidon / Coherence integration after first briefly reviewing the integration points between the two technologies.
Helidon and Coherence Integration
Helidon MP has been closely integrated with Coherence since June 2020. As shown in following diagram from that timeframe, the integration points include the CDI, Config, Health, Metrics, and Open Tracing APIs comprising the Eclipse MicroProfile specification.
The bootstrapping, CDI support, and Configuration integration points in particular lend to tremendous developer productivity. By simply launching a Helidon JVM, Coherence is bootstrapped and ready to serve data by the time Helidon-based microservices in the JVM are ready to accept requests. Coherence NamedMaps (caches) can be injected into Helidon services via CDI, and Coherence can be used as a configuration source for Helidon services.
For production operations, the Health, Metrics, and Open Tracing integration points provide state-of-the-art observability for applications built with the Helidon/Coherence combination.
Since the first release of the Helidon/Coherence integration, the MicroProfile specification has evolved, as have the Helidon and Coherence products. The following table captures the version compatibilities between the three.
Ease of Development with Helidon / Coherence Integration
Oracle’s FSGIU enjoyed easy and productive development of the Transaction Balance Engine (TBE) in OBACS by leveraging multiple Helidon, Coherence, and integration features focused on developer productivity:
- Automatic bootstrapping within Helidon JVMs of Coherence cluster members, and injection of Coherence caches into REST resources
- Helidon microservice configuration via MicroProfile Config
- Helidon’s support for OpenAPI
- Helidon’s support for scheduling
- Helidon’s Kafka Connector
- Coherence features for caching, transactions, grid-side events, and data source integration
The first of those features, automatic bootstrapping, deserves further description. Helidon-based microservices run in JVMs launched on a Helidon main class. When using Helidon/Coherence integration, Coherence is a class library loaded into such JVMs from a jar file, and a storage-enabled Coherence cluster member is automatically bootstrapped in each JVM before it accepts REST requests. The bootstrapped member joins an existing Coherence cluster, or starts one if none is discovered.
Thus as more Helidon JVMs are started, they expand the size and data storage capacity of the Coherence cluster, and obtain instant access to fault-tolerant key/value storage of shared Java objects in the Coherence data grid. The following diagram illustrates these concepts.
By leveraging the developer productivity features of Helidon, Coherence, and their integration, Oracle’s FSGIU team was able to develop the TBE microservice with 61 REST API paths, some supporting multiple HTTP methods, in a 28:37 ACID read:write ratio, within a half-year from start of development to scalability testing. And this in one of the world’s most complex and transactionally rigorous problem domains: accounts and transactions within the banking industry. Elapsed time from start of development to product launch was a little over a year.
Breakthrough Performance and Scalability
As illustrated in the following graph, the Transaction Balance Engine within OBACS exhibits linear scalability at 300 TPS per commodity CPU, and has been scale-out tested to 72,130 TPS, with a response time of a few milliseconds at the REST APIs comprising the TBE microservice. It achieves that performance and scalability by leveraging advanced Helidon and Coherence features for scale-out architecture, grid computing, transaction processing, and data source integration.
Coherence has been explicitly designed for horizontal scalability of data storage and processing capacity since it first created the In-Memory Data Grid technology space in 2001. The integration of Helidon and Coherence, allowing Helidon JVMs to double as storage-enabled Coherence cluster members, lets TBE leverage Coherence grid-side events within Helidon JVMs to implement advanced event-driven architectures adding to the performance and scalability of the solution. Using the unique partition-level transaction feature in Coherence, by which writes to multiple key-value pairs can be atomically committed in a single phase, is another major advantage in the TBE architecture. Partition-level transactions are part of Coherence’s in-place processing feature set, which pre-dates and is still superior to Redis Lua scripting for concurrency and data integrity.
Helidon’s Kafka Connector allows TBE to stream events into the environment, and Coherence’s asynchronous database writing allows TBE to back up banking account transactions and balances in a traditional database without suffering database write latency in REST API call response times.
Redis lacks these eventing, transactional, and data source integration features found in Coherence. So, such a scalable and performant solution could not have been built with Redis.
Maximum Availability Architecture in the Cloud
Oracle Banking Cloud Services is obviously cloud-native, designed to run on Oracle Cloud Infrastructure (OCI) using Container Engine for Kubernetes (OKE) and the Coherence Operator. This includes deployment to OCI Dedicated Regions in customer data centers. OCI regions have one or more Availability Domains (AD), which are isolated from each other, fault tolerant, and very unlikely to fail simultaneously. Availability domains don’t share infrastructure such as power or cooling, or internal networks, so a failure at one availability domain within a region is unlikely to impact other availability domains in the same region.
For maximum availability in OBACS production deployments, the cluster of Helidon/Coherence JVMs can be stretched across multiple availability domains in the same OCI region. Stretching the cluster across three ADs in a region provides the best data reliability, as Coherence automatically creates backup copies of data in a separate AD from primary copies, and using three ADs maximizes the reliability of Coherence-managed data in the event of an AD failure. Such a deployment architecture is illustrated in the following diagram.
OBACS can even be deployed in an active/active multisite architecture when using mulitple dedicated regions each having only one availability domain. Assuming a low-latency, high-bandwidth network between customer data centers, e.g. a metropolitan area network scenario, Oracle solution architects have proven that a Coherence cluster can span multiple OKE clusters by using Cilium, a Cloud Native Computing Foundation graduated project.
Get Your Own Flock Flying
This technology stack — Helidon integrated with Coherence deployed across multiple sites — enables industry-leading productivity, performance, scalability, and availability for mission-critical microservice architectures.
The banking industry has noticed the breakthrough performance and scalability enabled by a microservices architecture based on the integration of Helidon and Coherence. Since launching Oracle Banking Cloud Services, Oracle’s Financial Services Global Industry Unit has been honored to earn significant business wins with some of the world’s largest banking institutions.
You can enjoy similar success at scale with such an architecture. But not with Redis, which lacks the sophisticated features required for such a system. Instead, investigate the integration of the Helidon microservices framework and the Coherence data grid as a solution to your most challenging problems of developer productivity and solution scalability, in any problem domain. Refer to the following resources to get started:
- Java Magazine article on building Helidon-based, Coherence-backed RESTful microservices
- Helidon Sock Shop, an open source example application showing how to leverage Helidon/Coherence integration
- The Helidon Project website
- The Coherence Community website
- Coherence documentation on MicroProfile support
Special thanks, credit, and recognition to John Suresh and Loganayagi Balasubramanyam of Oracle’s FSGIU for architecting and developing the amazing Oracle Banking Accounts Cloud Service featured in this story.
