Optimistic Caching pattern in Typescript

Implement a high availability API with asynchronous data refreshing

Photo by Hello I’m Nik 🎞 on Unsplash

Why?

The kind of caching we will talk about is necessary to gain performance by not re-computing data, nor by re-calling web services.

Sometimes concurrency has to be taken into consideration when a single web service is called multiple times before it can respond once.

We will not talk about where to cache data, but mostly on how to reduce response times by adding an application layer which could seem quite non-academical.

Godspeed Rebels 🚀

How?

The goal of Optimistic Caching is to immediately return the latest cached data and, eventually, refresh the data server-side without keeping the client connected.

This is almost how the “stale-while-revalidate” HTTP’s header works. 🙋‍♂️

States

We distinguish multiple states for each entity:

• The data: It could be not-cached, expired or fresh
• The process (request + computing): It could be in progress or not

Algorithm

According to the previous cases, we will see how to process the request to deliver a response as soon as possible.

Case 1: The data is not cached

This situation happens only once during the very first request.

Case 2: The data is expired

If the data is cached but not fresh, the refresh process will be executed asynchronously to respond as soon as possible.

Case 3: The data is fresh

This is the simplest case because we just have to return the cached data.

Implementation

const ttl: number = 3_600_000 // 1 hour

let data: Data = null
let updatedAt: number = 0

async function getData(): Promise<Data> {
  const hasData = Boolean(data)
  const isFresh = Date.now() - updatedAt < ttl

  // Case 3

  if (isFresh) {
    return data
  }

  const deferred = dataService.getData().then((result: Data) => {
    updatedAt = Date.now()
    data = result

    return data
  })

  // Case 2

  if (hasData) {
    return data
  }

  // Case 1

  return deferred
}

Concurrency

Of course, this kind of behaviour could suffer from concurrency issues. ⚔️

In a naive implementation, if two or more clients request the same data (not cached or expired), the server will have to retrieve and compute the data for each request.

In a more clever implementation, we will use a lock (like a Mutex) to ensure that the retrieval data process will be executed once for the same query.

Case 1: The data is not cached

In the case of the data has to be fetched for the first time, the algorithm will be more complex but interesting.

In this case, the first request will wait for a result, whereas the concurrent incoming requests will wait for the end of the first request and retrieve its result.

Case 2: The data is expired

In the case, that the data is not fresh but already in the cache, the server will return the cached data to all incoming requests but it will refresh the data following the same mechanism as the one described for the first case.

Implementation

const ttl: number = 3_600_000 // 1 hour

let deferred: Promise<Data> = null
let data: Data = null
let updatedAt: number = 0

async function getData(): Promise<Data> {
  const hasData = Boolean(data)
  const isFresh = Date.now() - updatedAt < ttl
  const isFetching = Boolean(deferred)

  // Case 1

  if (isFetching) {
    return deferred
  }

  if (isFresh) {
    return data
  }

  // Case 2

  deferred = dataService
    .getData()
    .then((result: Data) => {
      updatedAt = Date.now()
      data = result

      return data
    })
    .finally(() => {
      deferred = null
    })

  if (hasData) {
    return data
  }

  return deferred
}

Error Management

Error management is the most specific part of this because it depends on your use cases.

In most cases, you will not need to handle incoming errors and your app should keep working to get the previously computed data.

But sometimes, your app has to handle these errors. For instance, if the error happens at the first loading, you should probably return the error to inform the front-end of the situation which will be fixed as soon as your data source will return a valid result.

This is almost how the “stale-if-error” HTTP header works. 🙋‍♂️

Another interesting error management could be to increase the TTL while the error keeps happening. It will decrease some useless computing.

In any way, you have to log the error because it only happens asynchronously (except for the first time).

Implementation

interface DataResponse {
  data: Data | null
  error: Error | null
}

const ttl: number = 3_600_000 // 1 hour

let deferred: Promise<Data> = null
let data: Data = null
let error: Error = null
let updatedAt: number = 0

async function getData(): Promise<DataResponse> {
  const alreadyHasData = Boolean(data)
  const isFresh = Date.now() - updatedAt < ttl
  const isFetching = Boolean(deferred)

  if (isFetching) {
    return deferred
  }

  if (isFresh) {
    return data
  }

  deferred = dataService
    .getData()
    .then((result: Data) => {
      updatedAt = Date.now()
      error = null
      data = result

      return data
    })
    .catch((result: Error) => {
      error = result
      data = null
    })
    .finally(() => {
      deferred = null
    })

  if (alreadyHasData) {
    return {
      data,
      error,
    }
  }

  return deferred
}

Let’s Cache 🙈

… and enhance performance!