Architecting API Management Solutions with WSO2 API Manager

Designing the Solutions Architecture, Planning Capacity Requirements, Designing the Deployment Architecture and Infrastructure Selection

At WSO2 we work with enterprises around the globe on implementing API management solutions for various scenarios. This includes use cases starting from managing APIs used within organizations, managing APIs exposed to business partners, managing third-party APIs consumed by internal systems, managing APIs publicly exposed to the internet, and many other business specific requirements. Organizations that require APIs for internal and partner usage may preliminarily deploy such systems on their internal data centers. Some deployments may also span to public cloud infrastructures depending on the need for scalability, geographical distribution, disaster recovery, cost savings, etc. On the other hand, businesses who run their existing systems on public cloud infrastructures may deploy API management solutions on the same platform.

In any of the above scenarios we first start with the business use cases and design a high level solution architecture which could illustrate the internal systems involved, integrations among multiple systems, messaging channels required, external services used, mediations needed, identity and access management, analytics, monitoring and every other aspect of the solution needed for exposing and managing APIs at different levels of the organization. This process would allow all stakeholders of the project to understand the high level requirements of the solution for supporting business needs and design the detailed solution architecture accordingly.

Designing The High Level Solution Architecture

Figure 1: A Sample High Level Solution Architecture

Most large organizations would have a collection of systems for managing information of employees, finance, inventory, manufacturing, distribution, maintenance, etc depending on the nature of their business. Developing organization-wide client applications to let their internal users interact with business functions may need to consume services from multiple systems either by invoking them directly or by producing integrated services. In addition, these internal systems may also need to talk to third-party systems such as Sales Force, Google Apps, Concur and JIRA using organization level API subscriptions. Moreover, such organizations may also need to expose APIs to the external world for allowing their partners, and third-party application developers to consume their business services for providing value-added services.

The high level solution architecture illustrated in figure 1 has addressed all of the above requirements for designing such solution. A collection of internal systems have been exposed via a central API management solution while some systems have been integrated together using an integration system. Communication channels with external systems have also been identified and mentioned on the left and right sides. The consumption of public APIs by external client applications has been shown on the left side and usage of external APIs for implementing internal business functions has been shown on the right. The external APIs have also been routed through the central API management solution without them being directly integrated with the internal system for better management.

According to this architecture, internal applications can consume services exposed by internal systems, internal integration system, and third-party systems via a central API gateway. At the same time, external client applications can also subscribe and consume organization’s public APIs.

Designing The Detailed Solution Architecture

Figure 2: A Sample Detailed Solutions Architecture

Once the high level solution architecture diagram is in place, enterprise architects may design a detailed solution architecture diagram for visualizing the actual interactions between systems, defining services, protocols, messaging channels, listing APIs, communication channels between internal networks and external networks, etc. The integrations between systems can be implemented according to Enterprise Integration Patterns (EIPs) using an ESB or a lightweight integration services platform such as BallerinaLang. Internal and external communication channels can be isolated by introducing a separate API gateway for public facing APIs. This approach would also allow each API gateway cluster to be scaled independently according to their own capacity requirements.

Moreover, all of the above components may need to be integrated with organization-wide systems which provide identity & access management, analytics, and monitoring capabilities. The importance of having such centralized systems is that when the business grows it would be difficult to manage users, permission models, analyze data and monitor production deployments if they are scattered. Most importantly, the infrastructure platforms required for each system and their geographical distribution would need to be carefully designed. These decisions may directly affect API latency and ultimately user experience depending on the latencies between data centers, the time taken to replicate databases, filesystems, etc. Therefore, it would be quite important to note down that information at this stage the project when designing an API management solution.

Planning Capacity Requirements

The second key phase of designing an API management solution is to identify the throughput required by each API manager component. The convention of calculating throughput is by the number of requests that a component can handle for a second. This is also known as transactions per second (TPS).

WSO2 API Manager consists of six main components; API Gateway, Publisher, Store, Key Manager, Traffic Manager, and Analytics. In almost all the scenarios API Gateway would get the highest number of requests. For each API request received by the API Gateway, a statistics message will be published to the Analytics server, and API usage data will be published to the Traffic Manager. Key Manager will only get access token management requests from the API Gateway when those cannot be satisfied using the cache. Similarly, the UI components would not get a considerable amount of load if the API portal is not exposed to a wider audience for utilizing the API management solution as a service. Otherwise, web UIs will only be used for managing API subscriptions and administrative purposes.

WSO2 research team together with the API manager team conducts regular performance benchmark tests for WSO2 API Manager for identifying the throughputs that it can handle according to different message sizes, backend latencies, and concurrent users. The results of the latest test carried out for WSO2 API Manager v2.1.0 can be found below:

Figure 3: Throughput comparison of WSO2 API Manager v2.1.0, Reference [1]

The performance test shown above has been done on AWS by using a compute optimized EC2 instance (c3.xlarge) which has 7.5 GB of memory and 4 vCPUs. The backend service latency has been defined in the graphs as “Sleep Time”. The JVM heap size of API manager has been set to 4 GB and it has been started in all-in-one mode by including all components in a single JVM. Ideally, when API manager components are decomposed each component may perform better than shown in these graphs when dedicated resources allocated for each.

According to the third graph, one instance of API Gateway can handle around 3000 TPS for an API which receives nearly 10 KB of data in the payload, which does not include any mediation policies and when the backend service takes nearly 500 ms to respond. Similarly, an API which contains mediation policies would be able to handle around 900 TPS depending on the complexity of the mediation policies used. Nevertheless, when the message size increases to around 100 KB, the throughput may decrease to around 400 TPS with the same parameters.

In general, an API Gateway would contain a collection of APIs which may handle different sizes of messages and communicates with backend services with various latencies. Moreover, the backend latency may also vary based on the request parameters. Therefore, it is important to note that these figures can only be used for taking a rough performance figure and when implementing an API management solution for a specific requirement a performance benchmark test might need to be executed for identifying the actual throughput figures that it may need to handle in production.

Designing The Deployment Architecture

WSO2 has defined five deployment patterns for WSO2 API Manager based on the following four aspects of the deployment:

  • Throughput required by each API manager component
  • Dynamic scalability of each component
  • Internal/external API gateway separation
  • Hybrid Cloud deployments

Once the solution architecture and capacity planning tasks are completed the deployment architecture of an API management solution can be designed by considering above requirements of the project.

Deployment Pattern 1

Active/Active All-In-One API Manager with High Availability

Figure 4: WSO2 API Manager Deployment Pattern 1

The deployment pattern 1 provides the simplest approach for deploying WSO2 API manager in production with high availability and least number of instances. As shown above, it includes two all-in-one API manager instances and two API manager analytics instances. Two dedicated databases will be needed for persisting API manager and analytics data. Sharing the same database among analytics and API manager instances may degrade the performance of the API Gateway. The organization’s LDAP based user store can be integrated for allowing single-sign-on (SSO) to UI components and authenticating and authorizing APIs. A shared volume would be needed for synchronizing API files and throttling policies among the API manager instances. This volume would need to have multi-write support and ability to mount to multiple instances. For an instance, on AWS an EFS instance and on on-premise datacenter an NFS server can be used for this purpose. In addition, the analytics instances would need instance volumes for persisting their individual index data and node identifier on the filesystem.

It is important to note that in this design API manager and Analytics instances would not be able to dynamically scale on demand due to the way their internal components have been designed to work. This is mainly due to the client-side failover transports Apache Thrift and AMQP used by the Traffic Manager, and Analytics components. If new instances are created at runtime, those will not be discoverable by the existing instances. Therefore, if dynamic scalability is required such solutions would need to switch to deployment pattern 2. The average throughput supported by this pattern can be calculated by referring the performance benchmark test results [1]. In general, this pattern will be able to support around 1500 TPS for APIs carrying 10K messages and around 700 TPS for 100K messages.

Deployment Pattern 2

Dynamically Scalable API Gateway & Key Manager

Figure 5: WSO2 API Manager Deployment Pattern 2

The second API manager deployment pattern has been designed for API management solutions which require higher throughputs at the API Gateway. In contrast to pattern 1, in this pattern, the API Gateway and Key Manager components have been taken out from the all-in-one instance and deployed separately for dynamic scalability. The rest of the components; Publisher, Store, Admin, and Traffic Manager runs in a single JVM using two fixed instances. Scaling the Gateway instances would also need to scale the Key Manager instances for supporting the access token requests. Theoretically, the two Analytics and Traffic Manager instances will be able to handle the load of ten API Gateway instances. Therefore, the API Gateway and Key Manager instances can be dynamically scaled until their instance count reaches ten. If the API Gateway instance count goes beyond ten, either Analytics and Traffic Manager instances would need to be manually scaled or deployment pattern 3 might need to be used.

Technically, the throughput supported by the deployment pattern 2 will be almost similar to the pattern 1 when the number of API Gateway instances are at minimum level. Nevertheless, those can be dynamically scaled using the infrastructure features such as an autoscaling group on AWS and Horizontal Pod Autoscaler on Kubernetes based on the incoming request load. At the minimum instance count, each component will provide high availability. The API Gateway, Key Manager and single JVM instances which includes Publisher, Store, Admin and Traffic Manager components would need layer a 7 load balancer for routing HTTP traffic. Apache Thrift and AMQP connections will be made point-to-point and those will not be able to load balanced.

Deployment Pattern 3

Fully Distributed API Manager Deployment

Figure 6: WSO2 API Manager Deployment Pattern 3

The deployment pattern 3 has been designed for solutions which require higher throughputs at the both API Gateway, and UI components. It deploys all API manager components individual clusters to be able to scale independently. Out of these six clusters, except for Analytics and Traffic Manager, all other components will be dynamically scalable based on the incoming request load. Analytics and Traffic Manager instances would need to be manually scaled and their dependent components would need to updated accordingly when needed. In this architecture, a layer 7 load balancer will be needed for API Gateway, Traffic Manager, Publisher/Admin, and Store.

Similar to deployment pattern 3, this approach will also provide an almost similar throughput at the API Gateway at the minimum instance count. The decision for moving to this pattern may need to be taken based on the throughput required by the API Gateway or dynamic scalability of the UI components.

Deployment Pattern 4

Internal/External API Gateway Separation

Figure 7: WSO2 API Manager Deployment Pattern 4

The deployment pattern 4 has been designed for solutions which require internal and external API Gateway separation. It has been designed by using deployment pattern 2 and by adding two additional Gateway instances. Therefore, it comprises of the same characteristics of pattern 2 except for the dedicated Gateway instances for managing external API traffic. Multiple API Gateway clusters can be added to WSO2 API Manager deployments using its gateway environment management feature. APIs can be specifically published to each gateway environment based on their accessibility level. Routing to each API Gateway cluster can be controlled at the network layer by only allowing external API traffic to reach the external API gateway instances.

Technically, the performance of this deployment pattern would also be similar to the deployment pattern 2 at the minimum instance count. Nevertheless, due to the internal/external API traffic separation, there will be more room in each API Gateway cluster to handle more traffic and those will be dynamically scalable.

Deployment Pattern 5

Hybrid Cloud API Manager Deployment

Figure 8: WSO2 API Manager Deployment Pattern 5

The deployment pattern 5 has been designed for solutions which require API management capabilities in both private and public cloud environments. As a reference, the on-premise deployment has been designed using deployment pattern 2. However, it can be designed using any of the above deployment patterns according to the requirements of the solution. The public cloud API management deployment can either use shared WSO2 API Cloud or a dedicated deployment of WSO2 API Manager in any of the organization’s own public cloud infrastructure. This type of a deployment can be used for different purposes:

  1. Utilizing Public Cloud as a Disaster Recovery (DR) Option

If the requirement is option 1, two data centers can be used in Active/Passive mode by replicating databases and filesystems from one data center to the other periodically. Traffic can be routed to the main data center always using a global DNS based load balancer such as AWS Route 53 and it can be switched to the secondary data center automatically if the main data center becomes unavailable.

2. Managing the Separation of Internal and External APIs

If the requirement is to have a separation between internal and external APIs and external APIs need to be exposed to the internet or systems available on a public cloud the two data centers can be used in Active/Active mode. In this approach, the global load balancer may need to configured to route requests based on the APIs using context path of the URL.

3. Obtaining Additional Infrastructure Resources

This approach would be quite challenging to implement if the latency between the on-premise data center and public cloud network is high and the connection between the data centers is not highly reliable. In this model, if API requests are routed to both data centers at the same time data might get inconsistent due to the time needed for replicating the state being high. One way to solve this problem would be to route requests based on the origin of the request without routing to both data centers at the same time.

Infrastructure Selection

WSO2 API Manager can be deployed on any virtual machine based platforms or on Kubernetes, OpenShift, and DC/OS container cluster management platforms which has following features:

  • A production-grade RDBMS: MySQL v5.6/5.7, MS SQL v2012/2014/2016, Oracle v12.1.0, PostgreSQL v9.5.3/9.6.x
  • Shared persistent volumes with the multi-write capability
  • Instance persistent volumes
  • Internal TCP and HTTP routing
  • Layer 7 load balancing for external routing

Instructions for installing on standard virtual machine based platform based on deployment pattern 1 can be found on WSO2 documentation. AFAIK currently instructions for other deployment patterns are not in place. Resources for installing on Pivotal Cloud Foundry and AWS are currently being implemented in WSO2 API-M PCF and WSO2 API-M Cloud Formation Github repositories. Configuration management modules for Puppet can be found in WSO2 API-M Puppet Github repository.

Resources and instructions for installing on Kubernetes and OpenShift platforms based on all five deployment patterns can be found in WSO2 API-M Kubernetes Github repository. Resources for installing on DC/OS are now being implemented in the WSO2 API-M DC/OS Github repository.


Architecting an API management solution using WSO2 API Manager may involve several steps for identifying the business requirements, designing the high level solution architecture, the detailed solution architecture, and the deployment architecture based on the infrastructure selection. WSO2 has identified five different deployment patterns for WSO2 API Manager to be deployed in enterprise environments based on performance, dynamic scalability, internal/external API separation, and Hybrid cloud deployment requirements. It can be deployed on any virtualized platform or container cluster manager which has production grade RDBMSs, shared and instance persistent volumes, internal HTTP and TCP routing, and layer 7 load balancing support for external routing.


[1] WSO2 API Manager Performance and Capacity Planning, WSO2:

[2] Securing Microservices (Part 1), Prabath Siriwardena:

[3] WSO2 API Manager Deployment Patterns, WSO2:

[4] WSO2 API Manager Customer Stories, WSO2:

[5] Compatibility of WSO2 Products. WSO2:

[6] Benefits of a Multi-regional API Management Solution for a Global Enterprise, Lakmal Warusawithana: