WSO2 APIM : The Effect of Maximum Number of Key Validator Client Objects (MaxIdle) on the Performance

Introduction

Image credit [1]

WSO2 API Manager is a complete solution for designing and publishing APIs, creating and managing a developer community, and for securing and routing API traffic in a scalable way. WSO2 API manager has 4 main components 1) API Gateway (GW), 2) API Key Manager (GW) 3) API publisher and 4) API Store. API Gateway secures, protects, manages, and scales API calls. It intercepts API requests, applies policies such as throttling and security using handlers and manages API statistics. API Key Manages the client, security and access token-related operations. The Gateway communicates with the Key Manager to check the validity of tokens, subscriptions and API invocations. The development and management of APIs are done using the API publisher (which is a Web interface). The API store provides API publisher to host and advertise their APIs and for API consumers to self register, discover, evaluate, subscribe to and use secured, protected, authenticated APIs. A detailed description of these components can be found here.

When the API Gateway receives API invocation calls, it contacts the API Key Manager service for verification (given that caching is not enabled at the Gateway level or the requested information is not found in cache).

The communication between API Gateway and Key Manager can happen in 2 ways: 1) Thrift call and 2) Web Service call. When communicating with the KM node, GW borrows a key validation client object from object pool (key validation object pool). The maxIdle parameter (refer to the section “key validation pool” ) is one of the core parameters of this pool and the value of this parameter has a significant impact on the APIM performance.

The objective of this blog is to investigate the effect of maxIdle parameter on the performance of the APIM performance. In this blog, I will provide the results when using a web service call for the communication between the GW and KM (note: I will add the results for thrift in the near future)

Key validation thread pool

Let’s have look at the implementation details of the key validation thread pool in a bit more detail.

The pool used in APIM for the key validation is aStack-based ObjectPool. Its implementation based on the Apache Commons is an Apache project

The stack-object pool is created using the following method.

public StackObjectPool(PoolableObjectFactory<T> factory,
int maxIdle,
int initIdleCapacity)

This above method creates StackObjectPool using the specified factory to create new instances, capping the number of "sleeping" instances to maxIdle, and initially allocating a container capable of containing at least initIdleCapacity instances. The initIdleCapacity parameter specifies the initial size of the underlying container, which can increase beyond this value if maxIdle > initIdleCapacity.

Configuration

Switching to WSClient

Let’s now have a look at how to configure APIM to use WSClient as the key validation client. Note that switching to WSClient is particularly important in a HA set-up to facilitate load balancing.

In <API Manager>/repository/conf/api-manager.xml look for <APIKeyValidator>.

The default key validation type as illustrated below is thrift.

<KeyValidatorClientType>ThriftClient</KeyValidatorClientType>
<EnableThriftServer>true</EnableThriftServer>

Change the above two values to WSClient and false. The entries should look like following after the change

<KeyValidatorClientType>WSClient</KeyValidatorClientType>
<EnableThriftServer>false</EnableThriftServer>

MaxIdle Parameter

In <API Manager>/repository/conf/api-manager.xml look for <APIKeyValidator>.

<ConnectionPool>
<MaxIdle>100</MaxIdle>
<InitIdleCapacity>50</InitIdleCapacity>
<ConnectionPool>

maxIdle: maximum number of idle (sleeping) client objects

InitIdleCapacity: initial size of the pool

If the above parameters has not been specified (i.e. commented out in the api-manager.xml) the following default values will be used:

maxIdle = 50 InitIdleCapacity=20

Deployment architecture

The performance results presented in this blog has been obtained under the following distributed set-up (note: total 6 nodes including the work-load generator). The duration of the test was 20 min (with a warm-up period of 10 min)

The performance results have been obtained by under the following conditions:

  1. APIM Version: APIM 2.0.0
  2. Cache configuration (scenario 3): GW: cache disabled, KM: cache enabled, JWT: disabled, note that this configuration has low performance compared to GW-cache enabled and KM cache disabled configuration because for each and every API call that hits the API Gateway, the Gateway issues a call to the key manager server. If the cache entry is available in the key manager server, it is returned to the Gateway. Otherwise, the database will be checked for the validity of the token The advantage of this method over the other is that we do not have to store any security-related information at the Gateway side.
  3. Throttling enabled
  4. Number of tokens = 10100
  5. Back-end service: echo service
  6. . Number of nodes 6 (8 GB RAM, 4 Cores, OS: Ubuntu )

Throughput

Let’s now have a look at how the throughput varies with increasing maximum number of idle objects under different concurrency levels. The results presented in blog has been done after setting the InitIdleCapacity (refer to “Configuration” Section) at 25, i.e.

<InitIdleCapacity>25</InitIdleCapacity>

The following figure shows the variation in throughput with the increasing maxIdle, (i.e. maximum number of key validator objects). Note that the under high concurrency (e.g. 200), specifying small maxiIdle can result in poor performance. For example, when concurrency is 200, if we use maxIdle value of 25, the maximum throughput we get is 624 requests/second.

Latency

Let’s now have a look at how the average latency and the standard deviation of the APIM varies with the maximum number of key validation clients objects.

The following figures shows the how the latency varies with the increasing number of MaxIdle (maximum number of key validator client objects) under two concurrency levels: 100 and 200. Note that if MaxIdle is significantly less than the concurrency level, then the average latency is significantly high and we also notice that increasing MaxIdle beyond the concurrency level does not result in any improvements in the latency.

Other interesting behaviour is the standard deviation of the latency is very high if the maxIdle is low (note: standard deviation is measure of predictability of response time as it measures the variability of latency values therefore the lower the standard deviation the better it is).

Discussion

Let’s now try to understand the behaviours that we observed above.

Recall that the initIdleCapacity parameter determines the initial size of the underlying pool, which can increase beyond this value if maxIdle > initIdleCapacity.

When the maxIdle << concurrency there is high probability that there is no existing idle object in the pool (immediately) available to popped from stack. Note that public T borrowObject() throws Exception method is used for borrow a object from the stack.

In such cases makeObject method of the pool's PoolableObjectFactory is invoked to create a new instance . The creation of object introduces a delay (waiting time) into request path. This results in the average latency to increase and the throughput to degrade. This delay also increases the variability of latency as it only applies to certain requests. As a result we will notice higher standard deviation in request latency when maxIdle is low (particular under high concurrency levels).

It is also worth noting that brrowObject() method has synchronized code blocks (to ensure only one thread can modify the state of the stack object pool) and as such there is a possibility of having thread contention under high concurrency levels which may affect the overall performance (note: behaviour can be resolved by tuning the configuration parameters of the stack object pool).

Conclusion

In this blog, I have looked at how the maximum number of key validator clients objects (maxIdle) affects the performance of APIM. We notice that specifying maxIdle to a lower (i.e. value can result in significant degradation in the performance. Therefore, it is important set maxIdle > maximum concurrency level. However, specifying maxIdle to a value which is (significantly) higher than maximum concurrency does not result in any additional performance gains.

http://commons.apache.org/proper/commons-pool/api-1.6/org/apache/commons/pool/impl/StackObjectPool.html#StackObjectPool(org.apache.commons.pool.PoolableObjectFactory, int, int)