Caching: Understating the scope

It’s not what you look at that matters, it’s what you see. — H.D. Thoreau

Most performance conversations, at one point or another, will get to the subject of caching. And for a good reason. Caching helps to hide some of the biggest performance inefficiency in many systems. That’s right, hide not fix. Whether it’s local storage of data, pre-calculated data set or a transient storage to reduce a number of server-side requests, all these methods are improving performance by limiting the frequency of expensive operations. Again, it is important to remember, cache does not improve the performance of the operations themselves, but rather reduces/delays/eliminates the need to execute those operations.

Don’t get me wrong, limiting a number of expensive connections or operations is important, this is why caching is a common component in most modern architectures. This is why it’s somewhat surprising to me that other good practice techniques that accomplish similar goals, but don’t fall under traditional definition of “caching”, often get ignored. Specifically, I am talking about scope caching™.

What is “scope caching”™?

Scope caching™ is something that most of you familiar with programming are likely already doing in some capacity. The simplest example of scope caching is a basic programming principle of not putting computational operations in a loop where possible. Running operations outside the loop, storing it in the variable and using a result variable in the following loop/code path is an example of reuse of pre-computed value. Which, in essence, what caching is trying to accomplish.

How many scopes are there?

Many. As many scopes as there are in a programming language and than some. That’s why it’s important to take time and understand the needed scope of cache requirements.

For example, scoping a variable globally is considered a bad practice. However, when working with a web language, global scope could be a webserver thread or webserver session. Loading something once and keeping it for the life of that scope for all the users can (and should) be completely acceptable solution. Pre-generated constants is a good example. If you (like many) store your countries and states in the database, loading them during a web server start and keeping them in global memory, instead of going to the database on every request, is a form of scope caching.

And speaking of web, REQUEST and SESSION are probably the two most common scopes that can be taken advantage of.

Consider a function that sits on top of auth service, that returns a current user object. It’s really simple. Useful. Widely used.

public function currentUser() {
  var currentUser = classes.getCurrentUser();
  return currentUser;
}

A lot of different modules and services take advantage of it to validate user’s access control, setting, etc. And because of that, it’s being called, at minimum, 14 times per page load. 14 requests to your database/cache/user service, over the network, to retrieve the exact same information on a single page. Service design ftw.

Problem? Of course. But an easily solvable problem, without the need to re-engineer the service.

public function currentUser() {
  if ( !isDefined("REQUEST.currentUser") ) {
    REQUEST.currentUser = classes.getCurrentUser();
  }
  return REQUEST.currentUser;
}

Storing a result of the first-retrieved user object in a REQUEST variable, persistent for the duration of that, well, request, will make the information available to different consumers on the same page without a round trip to the source.

Scope caching™!

The list of scopes above is not exhaustive, although it (hopefully) provides a good illustration of the diversity of scope levels and techniques to limit operations. Remember, if you need to ask for the information more than once, make sure to cache the response. (hey, it rhymed!)