Self-service gateways in an open-source culture
By Raj Kumar, Product Owner in the Integration Team at Macquarie Bank
Engineers in Macquarie’s Banking and Financial Services group (BFS) can now implement a gateway for their application in a day. Each use case that comes to this platform such as MicroGateways as a Service (mGaaS), comes as a tenant. The only requirements are basic skills in YAML and GitOps.
Today, Macquarie’s BFS group is a technology-driven retail bank with a multi-cloud approach, integrating both AWS and Google Cloud. The technology landscape continues to evolve and become more intricate, incorporating serverless architectures, cloud-based microservices and Commercially Available Off-the-Shelf (COTS) products.
API Gateways have emerged as pivotal components in streamlining communication between diverse services, ensuring security and performance while supporting the overall scalability and flexibility of the technology infrastructure.
In my colleague Ciaran Chan’s recent blog post, we presented our strategic approach of using micro-gateways to cater to our advanced API traffic management needs.
In this blog post, we’ll explore how we leveraged the power of GitOps and our expertise in micro-gateways to build a bank-wide self-service platform that empowers our engineers to solve complex API traffic management with ease.
Why self-service is key to the success of the platform
Traditionally, API gateways have been centrally managed. This meant engineers raised a request to a central team to onboard an API gateway and waited for weeks for it to be rolled out due to prioritisation demands, hindering the speed to market in BFS.
In order to prevent bottlenecks impacting development and deployment elements, it was essential that we built a self-service platform that allows development teams to independently manage and deploy their APIs and related services. We wanted to enable teams to take ownership of their API lifecycle: from design and development, to deployment, monitoring, and scaling, without relying on a central authority for every aspect of management.
A self-service micro-gateway platform offers several advantages, such as autonomy, flexibility, scalability, innovation and reduced administrative overhead.
MicroGateways as a Service (mGaaS)
The mGaaS platform offers a centralised infrastructure that provides various controls and functionalities, while still allowing service teams to manage their APIs independently. We leveraged the power of GitOps and used the “everything as code” approach when building this platform. This means treating all configurations, policies and infrastructure components as code. This approach enables version control, repeatability, automation and consistent zero-downtime deployments.
Paved path for the tenants
• Abstraction: In the mGaaS platform, we created an abstraction layer using a simplified interface. This enabled us to provide higher-level tools that shield individual service teams from dealing directly with the underlying micro-gateway product Gloo Edge, Kubernetes or Amazon Elastic Kubernetes Service (EKS). This abstraction layer provides a user-friendly, simplified and standardised interface for service teams to manage their APIs without requiring detailed knowledge of the underlying products. This promotes self-service, collaboration and consistency while also ensuring governance and security standards.
• Templates: The platform offers pre-defined API templates and profiles that service teams can choose from when defining their APIs. These templates encapsulate best practices, compliance and security requirements. This includes supported authentication mechanisms, authorisation policies, rate limiting configurations, circuit breaker configurations and canary deployment configurations.
• Plugins: Any feature that was not available out of the box in the micro-gateway product, like advanced mTLS, was developed as reusable plugins and made available as part of the platform offering. This allows service teams to mix templates and plugins when defining routing rules for their APIs.
Key Platform Components
• Infrastructure as Code (IaC): For the micro-gateway platform, we use AWS CloudFormation to define and manage the underlying infrastructure. This includes provisioning the servers, networking, load balancers, DNS names and any other necessary resources. For routing configurations, we use Helm charts to store the resource manifests.
• Micro-gateway: Micro-gateways are implemented using Gloo Edge API gateway. Gloo Edge is an envoy-based cloud native API gateway that handles API traffic, implements security measures and enforces access controls. Essentially, this is what drives the entire platform.
• Containerisation: We use Amazon Elastic Kubernetes Service (EKS) to host the platform. By using Kubernetes, we are able to build a consistent deployment environment that can scale easily.
• API security policies as code: Using Open Policy Agent to define policies as code allows us to apply standardised security policies consistently across all APIs.
• Continuous Deployment (CD): We use ArgoCD to implement continuous deployment. With ArgoCD, we are able to continuously monitor our deployed platform and compare the current state with the desired state as specified in our GitOps repository.
• IaC Review and approval process: Since everything is defined as code, any change to the platform or routing configuration is made via pull requests. We have built a fully automated GitOps bot that reviews all pull requests, adds relevant approvers for impacted APIs, performs code lint checks and enforces release control procedures.
Key Concepts
• Tenant: mGaaS is a multi-tenanted platform. Each team that wants to use the platform is onboarded to the platform as a tenant.
• Product: A logical group of APIs. A tenant can have multiple products.
• Route: Represents an API. This is the rule that instructs the micro-gateway where to route the API traffic to.
• Upstream: This is the actual API provider and encapsulates the host and port.
• Upstream Groups: A logical group of upstreams allows for weighted routing among upstreams.
• Downstream: This is the API consumer or client.
Modern gateway features as plugins
Our aspiration was to create a platform characterised by modularity and flexibility. Central to this approach is offering platform features as plugins. As the technology landscape evolves at an ever-accelerating pace, the ability to remain agile and responsive is paramount. By embracing the concept of plugins, we infuse the mGaaS platform with a level of modularity that is designed to not only meet the needs of today but also effortlessly adapt to the needs of the future.
Plugins enable a modular architecture where each feature is a separate component. This promotes code reusability, easier maintenance, and the ability to develop and update features independently.
Some of the key features offered by the platform are detailed below.
Canary traffic shifting
When releasing changes to an API, it is essential that we are able to control how those changes get exposed to users. This “progressive delivery” approach to releasing software allows you to reduce the impact when changes may be introduced unintentionally. Approaches for controlling the release of new changes by slowly exposing them include canary release, blue-green deployment, and traffic shadowing. Canary traffic shifting allows you to direct a certain percentage of traffic to a set of new backends, while the old backends serve the remainder of the traffic. This enables validating of a new backend stack or new version of the backend before opening it up to all the traffic. We combine backends into groups and then assign weights to the groups to govern how the traffic is split.
Traffic shadowing
With traffic shadowing, we make a copy of the incoming request and send that request out-of-band (that is, out of the real request path, and asynchronously) to the new version of the API. From there, we can simulate taking live production traffic without affecting real users. We can observe the behaviour of the new release, compare it against expected results, and determine whether to proceed with the release (such as by rolling out to real traffic with percentage-based canary) or roll it back.
Traffic shadowing with mGaaS makes a copy of an incoming request, proxies the real request to the appropriate backend (the normal request path), and sends the copy to another upstream. The copied message is ignored for failures or responses. In this case, you can deploy v2 of a service and shadow traffic to it in production without affecting user traffic. This ability to shadow is incredibly important because it allows you to begin your release or canary process with zero production impact.
This policy is applied per route.
Circuit breaker
We implemented a circuit breaker pattern, which helps maintain the response time of the system by quickly rejecting a request for an operation that’s likely to fail, rather than waiting for the operation to time out.
Circuit Breakers are defined per upstream.
Rate limiting
We then implemented rate limiting, which helps prevent a high volume of requests from overwhelming a server or API.
We used mGaaS to simplify the rate limiting model, allowing us to specify the number of requests per configurable unit of time that was made against individual routes.
Rate Limiting is applied per route.
Retries
The retry feature of mGaas allows the developer to define a retry policy for the route. The policy allows you to specify when to retry, how many times to retry, per retry timeout and retry backoff interval. This improves the reliability of the API.
Retries are configured per route.
Authentication and authorisation
As a platform that is offered bank-wide, we support Active Directory, mTLS and JWT authentication methods to cater to legacy applications, microservices and COTS products. Authorisation plugin means you can define the HTTP methods that should be allowed per API.
GitOps model
GitOps is a set of practices to manage infrastructure and application configurations using Git, an open-source version control system.
GitOps works by using Git as a single source of truth for declarative infrastructure and applications. GitOps uses Git pull requests to automatically manage infrastructure provisioning and deployment. The Git repository contains the entire state of the system so that the trail of changes to the system state are visible and auditable.
Repository structure
The consumers of the platform are onboarded as tenants. The platform configuration and tenant specific configuration is stored in a mono-repository in Bitbucket. The directory structure is as shown below. All manifests are stored in the form of Helm charts.
GitOps workflow
Our GitOps implementation uses the following tools from the Argo Project:
Argo Events
Argo Events is an event-driven workflow automation framework for Kubernetes which helps trigger things like K8s objects and Argo Workflows on events from a variety of sources like webhooks, S3, schedules, messaging queues, gcp pubsub, sns and sqs.
The main components of Argo Events are: EventSource, EventBus, Sensor and Trigger.
Argo Workflows
Argo Workflows is an open-source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).
We define an Argo Workflow template that uses a custom docker image to run the following steps:
• Consume the pull request metadata passed on by Argo Events Trigger
• Check if the pull request destination branch is a Production branch (such as main)
• If PR destination is a non-production branch:
- render helm manifests
- run manifest lint checks
-if all steps pass, merge the pull request
• If PR destination is a Prod branch, then:
- render helm manifests
- run manifest lint checks
- run release control procedures i.e. the PR should have a valid release ticket that is approved
- if all steps pass, merge the pull request
Once Argo Workflow merges the pull request, ArgoCD will sync the changes to the live cluster.
Argo CD
Argo CD follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state. Kubernetes manifests are specified using the helm chart format.
Argo CD continuously monitors running applications and compares the current, live state against the desired target state (as specified in the Git repository). As soon as Argo Workflow merges a pull request to the Git repository, the application goes into an Out-Of-Sync state. Argo CD then automatically applies the difference to live cluster to bring it in sync with the target state.
Inner-sourcing — everyone is part of the success story
We have adopted the open-source practices of software development where all engineers can collaborate and contribute irrespective of the team they belong to.
This opens up our platform code base within Macquarie so that it’s transparent, allowing broader contribution and usage while maintaining quality and vision of the platform.
Benefits of adopting inner-sourcing includes:
• Break-down silos
• Expand developer base
• Reduce dependencies
• Increase innovation across Macquarie
• Improving quality through transparency
• Increase engagement
Journey continues
At the core of our mission is a steadfast dedication to refining the mGaaS platform through tackling complex engineering issues. We recognise that growth and progress are fuelled by the identification and resolution of challenges, and this belief propels us forward. As we solve these challenges, we not only refine the platform’s capabilities, but also enrich our own knowledge base. This cycle of discovery, analysis, solution-building, and implementation underpins our journey towards a more robust and versatile platform. If you are interested in being on this journey with us, please visit our careers page.
