How to get the Top N most accessed keys in a Coherence cache?

Ever had to figure out what are the most accessed entries (hot keys) in an Oracle Coherence cache?

There are multiple ways to do this in Coherence, but there is a lesser known way (to me at least) to do it by leveraging readily available internal cache content tracking attributes within Coherence.

Let’s say you work for a wireless company and you are caching price plans for a speedy response time on their retail website. Upper management needs to know what price plans are the most requested and also keep track of the top 10 price plans on a daily basis.

One straightforward approach would be to track for each cache item a counter, and every time you access an item, you would increment the corresponding counter (and keep it in another specific cache). But you would need, in addition to the original price plan cache read, one read + one write access for the tracking counter. Alternatively you could decide to keep the counter within the price plan entity, but then things would be even worse since you would be making a read then a write for each price plan request. Remember, writes are slower than reads.

The better way is to fetch the internal touch count associated to a local cache entry. This assumes that your cache is configured to use a local cache as its backing map.

<cache-config>

  <caching-scheme-mapping> 
    <cache-mapping>
      <cache-name>PricePlan</cache-name>
      <scheme-name>PricePlanScheme</scheme-name>
    </cache-mapping>
  </caching-scheme-mapping>

  <caching-schemes> 
    <distributed-scheme>
      <scheme-name>PricePlanScheme</scheme-name>
      <service-name>PricePlanCacheService</service-name>
      <backing-map-scheme>
        <local-scheme>
          <expiry-delay>10d</expiry-delay>
        </local-scheme>
      </backing-map-scheme>
      <autostart>true</autostart>
    </distributed-scheme>
  </caching-schemes>

</cache-config>

Coherence internally keeps track of how many times an entry is touched (read or write). It needs that to manage cache eviction. You can read more about it here; to support the least frequently used eviction policy, Coherence internally keeps track of the number of times each cache entry is accessed.

So how are we going to fetch this value for each cache entry in the backing map and, better, figure out only the top N hot keys?

We will first need to visit each backing map of this cache, on each storage member (since we are using a partitioned cache). For this we will use an Invocable (HotKeyInvocable) through an InvocationService (InvocationService-TouchCount), to be added to your cache configuration:

<invocation-scheme>
  <scheme-name>invocation-service</scheme-name>
  <service-name>InvocationService-TouchCount</service-name>
  <autostart>true</autostart>
</invocation-scheme>

Here is what the run() method of this invocation handler looks like:

/**
 * Fetches the touch count of each entry of the backing map.
 */
public void run() 
    {
    SortedCollectionWithCapacity<HotKeyData<K>> results = 
        new SortedCollectionWithCapacity<HotKeyData<K>>(this.topN);

    DistributedCacheService cacheService =
        (DistributedCacheService) CacheFactory.getService(cacheServiceName);
    Member localMember = cacheService
        .getCluster()
        .getLocalMember();
    int localMemberId = localMember.getId();
    Set backingMapEntries = cacheService
        .getBackingMapManager()
        .getContext()
        .getBackingMap(cacheName)
        .entrySet();
    BackingMapContext backingMapContext = cacheService
        .getBackingMapManager()
        .getContext()
        .getBackingMapContext(cacheName);
    Converter converter = backingMapContext
        .getManagerContext()
        .getKeyFromInternalConverter();

    for (Object entry : backingMapEntries) 
        {
        LocalCache.Entry localCacheEntry = (LocalCache.Entry) entry;
        K key = (K) converter.convert(localCacheEntry.getKey());
        results.add(new HotKeyData<K>(key, localCacheEntry.getTouchCount()));
        }

    this.setResult(results);
    }

We are accumulating the partial results in a custom structure SortedCollectionWithCapacity to keep the top N keys in a sorted order (not discussed here, but bundled in the source code). The rest of the code is about how to retrieve the valuable touch count from the backing map.

We will run this Invocable against each storage node owning the the targeted cache service. Here is how we will invoke it:

// we will gather each result per member in this structure
// see InvocationObserver.memberCompleted
Map<Member, SortedCollectionWithCapacity<HotKeyData<Integer>>> invocationResults =
    new HashMap<Member, SortedCollectionWithCapacity<HotKeyData<Integer>>>();

DistributedCacheService distributedCacheService = 
    (DistributedCacheService) CacheFactory.getService("PricePlanCacheService");
Set<Member> storageMembers = 
    distributedCacheService.getOwnershipEnabledMembers();
InvocationService is = 
    (InvocationService) CacheFactory.getService("InvocationService-TouchCount");

InvocationObserver observer = 
    newInvocationObserver(countDownLatch, startTime, isVerbose());

for (Member member : storageMembers) 
    {
    HotKeyInvocable<Integer> invocable = 
        new HotKeyInvocable<Integer>(getCacheName(), 
            getCacheServiceName(), 
            getTopN());
    is.execute(invocable, Collections.singleton(member), observer);
    }

Note that we need to register an InvocationObserver (com.tangosol.net.InvocationObserver) to gather the execution result from each member once the local invocation completes:

/**
  * Create and return an InvocationObserver for observing 
  * the progress of asynchronous Invocable execution.
  */
private InvocationObserver newInvocationObserver(final CountDownLatch countDownLatch, final long startTime, final boolean isVerbose)
    {
    return new InvocationObserver()
        {
        public void memberCompleted(Member member, Object result)
            {
            invocationResults.put(member, 
                (SortedCollectionWithCapacity<HotKeyData<Integer>>) result);
            countDownLatch.countDown();
            CacheFactory.log(String.format("Task completed on %s.", member));
            }

        public void memberFailed(Member member, Throwable throwable)
            {
            invocationResults.put(member, null);
            countDownLatch.countDown();
            CacheFactory.log(String.format("Task failed on %s.", member));
            CacheFactory.log(throwable);
            }
        public void memberLeft(Member member)
            {
            invocationResults.put(member, null);
            countDownLatch.countDown();
            CacheFactory.log(String.format("Member left before task completed: %s", member));
            }

        public void invocationCompleted()
            {
            CacheFactory.log("invocation completed");
            }
        };
    }

Once all the Invocables have completed, we can access the results from each member, merge them (through the SortedCollectionWithCapacity class), and collect the final top N most accessed keys:

SortedCollectionWithCapacity<HotKeyData<Integer>> mergedResult =
    new SortedCollectionWithCapacity<HotKeyData<Integer>>(getTopN());

for (Map.Entry<Member, SortedCollectionWithCapacity<HotKeyData<Integer>>> entry : invocationResults.entrySet())
    {
    Member member = entry.getKey();
    if (entry.getValue() != null) 
        {
        SortedCollectionWithCapacity<HotKeyData<Integer>> result = 
            entry.getValue();
        mergedResult.merge(result);
        }
    }

This source code and a running example is available here.

Here is a sample execution output:

(Coherence 12.2.1.4.20, 3 storage nodes, 1 million entries, top 3 hot keys)

Execution on member Member(Id2, Address=127.0.0.1:9002, Role=cache-server) took 106 ms
Execution on member Member(Id1, Address=127.0.0.1:9001, Role=cache-server) took 117 ms
Execution on member Member(Id3, Address=127.0.0.1:9003, Role=cache-server) took 113 ms
Total gathering of top 3 hot keys took 150 ms
SortedCollectionWithCapacity [sortedSet=[[key=50, touchCount=50], [key=25, touchCount=25], [key=10, touchCount=10]], maxCapacity=3]

I hope this efficient way of finding and reporting the top N hot keys in Coherence caches proves useful to other Coherence users, some of whom have asked about such solutions.

Note from editors: we are pleased to publish this solution story from a longtime leading Coherence user in the community. If you’re reading this and have your own idea on a cool solution trick you’d like to share with the community of Coherence users, please do reach out to the editors of this publication.