Caffeine cache simplified in Kotlin
Always we need fast responses and don’t overburden services in an unnecessary manner. At this point we can think on use a cache.
Introduction
A cache can contains the values of a service response, like a database, a micro-service or a disk resource. Is an intermediary between us and the resource. It can hold the value in a time’s period that is define for us, after that period the value is obsolete and we obtain a new value of that resource. We need a key to identify the associated value in the cache, if it contains the value then returned it otherwise it go to the service, store it and return the value.
In the Fig 1 we can see that the cache exist inner the application. If we handle more than one node we would have one in each one. The concept is quite different to i.e. Memcached where it is a server independently of the application all nodes ask at that Memcached server. While Caffeine is in memory, a first level, Memcached would be in a server, a second level.
Caffeine permits handle sync and async (very interesting for corountines) population and the load could be manual or with get method. We will explain the sync manual method. I think that can include the mostly use cases.
The eviction is a fundamental part of the cache logic. This indicates when a key/value will be automatically removal in the cache. Exist three methods to evict entries size-based when is full start the eviction , time-based based in time and reference-based. In this article we are going to explain size-based with time-based policy.
Configuration
We need to import the library :
implementation 'com.github.ben-manes.caffeine:caffeine:3.0.5'A very simple configuration :
//configuration
private val cache: Cache<String, Item?> = Caffeine.newBuilder()
.maximumSize(1_000)
.expireAfterWrite(
60,
TimeUnit.MINUTES
)
.recordStats()
.removalListener { key: String?, _: Item?, cause ->
logger.info("Remove key : $key for the cause : $cause")
}.build()//point of entry
fun getItem(itemId: String): Item? {
return cache.get(itemId, retrieveItemFn)
}private val retrieveItemFn: (String) -> (Item?) =
fun(itemId: String): Item? = repository.getItem(itemId)//optional is only if you want get the stats by a controller for example
fun getStats(): Map<String, String> {
return mapOf(
"hitCount" to cache.stats().hitCount().toString(),
"loadCount" to cache.stats().loadCount().toString(),
"missCount" to cache.stats().missCount().toString(),
"missRate" to cache.stats().missRate().toString()
)
}
Let’s start with the configuration itself. We set a maximum of 1.000 entries with this will be sure that not grow in an unmanageable way. If the cache is full then try to evict entries that have not been used recently or very often.
About the time that an entry can lives we use expireAfterWrite(…) this mean that the value will be alive 60 minutes in the cache, but we also could have expireAfterAccess(…) or expireAfter(Expiry). But be careful if you use expireAfterAccess(…) in a multi node cluster the information could stay inconsistent between nodes. Think in the use case that an entry was stored 59 minutes ago so we receive a new request for that key. One node, the one who received the request, will have that value another 60 minutes because refresh the time to live. But the others nodes one minute lates will evict the entry and if receive a new request go to the database to get the updated data. And with expireAfter(Expiry) you can create a custom expiration policy or use some definitions like FixedExpiry.
We define a listener to log when an entry is evicted (or if it is removed manually too). This listener is executed when the entry is removed, the cache detect that an entry is expired when try to get the value in that moment evict the entry and go to obtain a new resource. This time can be more than the 60 minutes because de deleted is reactive and no proactive.
The last part are the statistics, we need explicit tell that want to record them. Otherwise they can’t be registered. In the method getStats() is an example that return some of the values. With this information you can create a controller to get it or better send it to a tool to create metrics.
Flow
Now we go to talk about the point of entry of this example : getItem(itemId: String). This method want to retrieve an item what key is receiving in the parameter, then call the cache with the key and passing a function that in the case that the key doesn’t exist go to the repository to obtain it. As we can see this semantic is the same that we explain in the Fig 1.
I hope that this simple explanation can help you in some way.
