API Caching — Why/ What and WSO2 API Manager
Caching is something which we hear a lot nowadays. Application developers, Internet Service Providers (ISPs), API Management/ Gateway vendors, Application Server and even Database Management vendors talk about Caching these days since it adds a lot of values into the business applications.
Applications that communicate across network boundaries, and applications that communicate to different backend systems via APIs heavily depend on caching. This is an area where we focus on API Caching.
Before we talk about why/ what Caching or API Caching, lets first look at a typical application behavior without caching. Below is the typical request/ response path for a mobile/ desktop/ web client which communicates across network boundaries:
In the above diagram, web/ mobile/ desktop user [1] makes a request, request then reaches the ISP infrastructure [2], and then it gets routed to the organization network [3]. There will be a hardware or software gateway [4] that will apply some initial filtering/ policy checking/ throttling/ etc., and then it routes the request to the middleware layer — application server [5]. The application server then will do some server-side processing, and make different API calls, and then APIs will communicate with the database [6] to fetch required data. Once it fetches the required data, then the response will travel all the way back to the user via the same path/ route.
To improve performance, Caching can be implemented at various or all the touchpoints. Caching can be implemented at the Client application end, at the ISP level, at the Gateway layer, at the Middleware/ Application Server layer, or at the Database layer. The choice of the layer in which caching will be implemented will vary depending on many parameters.
What are the benefits of using Caching?
Below are the main reasons why we need to consider implementing Caching:
- Improve the performance
- Improve the scalability
- Improve the throughput
It’s advisable to enable caching as much as closer to the client application/ end-user, so it will short-circuit the request/ response path by providing better performance and higher scalability and throughput.
Below diagram provides a conceptual overview of how we can short-circuit the request/ response path by enabling caching at different touchpoints:
As shown in the above diagram, depending on the location of the cache and depending on the validity of the cache (if the cache is not expired already), the request will take a short trip/ path to server the requested information to the client.
For example, let's assume that you have designed the initial application estimating a volume of 100 requests per second. And over time, with the increase of the demand, application experienced over 1,000 requests per second, and the database layer could handle only maximum of 500 requests per second, so it started rejecting the majority of the requests. To address this, you can take many architectural decisions and enable caching in one of the layers is one of them.
What to Cache?
When enabling caching, selecting what to cache is an important design decision that we have to make. Selecting the right content/ data for caching is vital, and this will have a direct impact on the behavior/ functionality of the application. Below are the key deciding factors when selecting what to cache:
- Frequency/ speed of change of the data
- Time sensitivity of the data
- Security of the data
Static content such as HTML/ CSS/ JavaScript/ JPEG/ PDF don’t change much frequently. They don’t carry many security concerns as well. So, these types of static data can be cached and can be cached for a longer period of time (days/ weeks).
Sensitive data such as stock rates are frequently changing and time-sensitive. Your application may call an API to fetch stock ratings every 5 minutes or less, so these data may not be cached at all or cached for a very shorter period of time.
In summary, when we want to leverage caching in our application, to improve the performance, scalability, and throughput, there are few key design decisions which we have to make:
- Which component(s) should control the caching? — either Client Application, ISP, Gateway, Application Server, Database or all of them
- What to cache? — depending on the frequency/ speed of change of the data, time-sensitivity of the data and security of the data
- Duration of the cache? — depending on the frequency/ speed of change of the data, time-sensitivity of the data and security of the data
When designing and developing APIs, it always recommended to use caching. API developers will use Cache Control Directives to control the caching behavior of the APIs. Cache Control Directives which are defined in the HTTP Specification RFC 2616 allow API designers/ developers to control things such as who can cache, the validity period of the cache, under what conditions the data can be cached.
If the APIs are designed and developed with caching enabled, having an API Gateway that supports caching will further improve the performance, scalability, and throughput of your application.
WSO2 API Manager provides caching at various levels. It provides below caching mechanisms:
- Gateway caching
- Resource caching
- Response caching
- Key Manager caching
- Developer Portal caching
Below conceptual architecture diagram shows the use of WSO2 API Manager within your application:
For more details, refer — https://apim.docs.wso2.com/en/latest/administer/product-configurations/configuring-caching/#configuring-caching
Response caching is the most commonly used caching mechanism when configuring caching in API Gateways. WSO2 API Manager uses WSO2 Enterprise Service Bus (ESB)’s cache mediator to cache response messages for APIs. Caching improves performance because the backend server does not have to process the same data for a request multiple times. You need to set an appropriate timeout period to offset the risk of stale data in the cache.
You need to enable response caching when creating a new API or editing an existing API using the API Publisher portal as shown below.
The default cache timeout is 300 seconds and it can be configured depending on the business requirements considering various parameters. We also can configure a few other properties related to cache behavior by making modifications to the configuration file — velocity_template.xml, located at:
M_HOME>/repository/resources/api_templates/velocity_template.xml
For more details, refer — https://apim.docs.wso2.com/en/latest/learn/api-gateway/response-caching/
