Photo by Matthew Henry from Burst

Promise, Thenable and Cancelable Timers in JavaScript

Back in March, our VP of engineering David Underhill wrote a post about choosing the right tools to build Pocket Gems’ next generation backend servers. Since then, we benchmarked a few more options and finally settled on AWS because it met all of our needs. I have also since joined project Todea and created some foundational building blocks, including a task queue service. While building that service, I encountered a rather interesting problem that required a combination of fairly advanced and tricky features of JavaScript to solve. I’d like to share what I learned along the way.

The Problem

A task queue service invokes some HTTP requests on a caller’s behalf asynchronously. For example, instead of sending a GET request to www.pocketgems.com/careers/ myself, I can have the task queue service do the work for me. With the help of the async & await keywords introduced in ES6, it is very straightforward to send HTTP requests to other servers on a user’s behalf:

await http.get('pocketgems.com/careers/')

We also needed to make sure our servers weren’t waiting indefinitely for a problematic HTTP request to finish. In other words, we needed a timer-based escape path that runs in parallel to the await statement above.

A Simple Timer

JS has a built-in setTimeout function to trigger a callback after a delay. It can be used like this:

setTimeout(callback, 1000) // Invoke callback after 1 second

To make setTimeout work nicely with our http request in the async & await world, we could wrap the setTimeout function into a Promise like below.

const future = new Promise((resolve, reject) => {
  setTimeout(resolve, 1000)
})
await future // Blocks program from reaching the next line for 1s

It’s a Race

Once we had a Promise-based timer, all we needed was an escape path to work in parallel to the http request. We used Promise.race() to achieve this behavior. Unlike Promise.all() which blocks program execution until all sub-promises are resolved, Promise.race() resolves when any sub-promise is resolved. So, it’s literally a race.

await Promise.race([
  http.get('pocketgems.com/careers/'),
  new Promise((resolve, reject) => { setTimeout(resolve, timeout) })
])

A Leaky Handle

Even after the code worked as intended, my work was not done. The unit test framework Jest started reporting that something was not finished after all test cases were run, which normally indicates a leaky handle. The term “leaky handle” refers to a situation where a handle to some system resource was acquired but never released. For example, when a file descriptor to a socket is opened but never closed, a leaky handle will result.

Adding --detectOpenHandles to Jest and adding require('leaked-handles') to unit test files helped with debugging these issues. Normally, a leaky handle is reported as:

tcp stream {
  fd: 0,
  readable: true,
  writable: false,
  address: { 127.0.0.1:9423 },
  serverAddr: 'https:localhost:9423'
}

This indicates some network component was not properly or promptly shut down. In our case, the output was something like below, where address and serverAddr were both empty.

tcp stream {
  fd: 0,
  readable: true,
  writable: false,
  address: {},
  serverAddr: null
}

Due to the lack of useful debug information, I had to bisect the problem by commenting out chunks of the task queue logic and unit tests code. The method call to setTimeout() turned out to be the root cause. It also, however, revealed a larger problem.

When we called:

await Promise.race([
  http.get(‘pocketgems.com/careers/’),
  new Promise((resolve, reject) => { setTimeout(resolve, timeout) })
])

and the http request was resolved first, the server was still left with a piece of logic that was scheduled to run sometime in the future to resolve the timer promise. NodeJS implements an event loop where all operations are done on a single thread but are executed in a somewhat round-robin fashion. If we had too many scheduled logic that served no purpose, other critical code paths could be undesirably delayed.

A Cancelable Timer

So, we had to cancel the scheduled timers. A naive approach would be calling clearTimeout with the handle returned by setTimeout.

const handle = setTimeout(callback, timeout)
clearTimeout(handle)

To fit this into our async & await world, the code would become something like:

let handle
await Promise.race([
  http.get(‘pocketgems.com/careers/’),
  new Promise((resolve, reject) => {
    handle = setTimeout(() => {
      handle = undefined
      resolve()
    }, timeout)
  })
])
if (handle) {
  clearTimeout(handle)
}

It is not elegant at all. I wished we could pull the timer logic out into a separate object, so we could keep the roughly original (and beautiful) code.

That was indeed possible with thenable objects, which are objects that implement:

then(resolve, reject) {}

They can be used the same way as a promise object. The method then is invoked when the thenable is awaited:

await thenable // Calls method `then`

So, I made this thenable class:

class Timer {
  constructor (timeout) {
    this.timeout
    this.handle = undefined
  }
  cancel () {
    if (this.handle) {
      clearTimeout(this.handle)
      this.handle = undefined
    }
  }
  
  then (resolve, reject) {
    this.handle = setTimeout(function () {
      this.handle = undefined
      resolve()
    }, this.timeout)
  }
}

And I updated the http request logic to be:

const timer = new Timer(timeout)
await Promise.race([
  http.get('pocketgems.com/careers/'),
  timer
])
timer.cancel()

Adding a Cherry On Top

To take things one step further, I wanted to remove the need to construct the timer with the keyword new every time. A simple wrapper method could work.

async sleep(timeout) {
  return new Timer(timeout)
}

The code above didn’t work, however, because when a value is returned from an async method, JS checks if the value is a Promise. If not, JS will automagically wrap the returned value in a Promise. I found this behavior documented here.

In our use case, we were returning a thenable object, which behaves like a promise but is still not a promise according to the ECMA 2015 spec, since a promise is detected with the following pseudo code:

IsPromise(x):
  If Type(x) is not Object, return false.
  If x does not have a [[PromiseState]] internal slot, return false.
  Return true.

When we called sleep(1000), we were getting a promise which resolves to our thenable Timer object that can be awaited on. Counterintuitively, we would have needed to use two await back-to-back like below.

const promise = sleep(1000)
const timer = await promise // This results in our Timer object
await timer // Waits for 1s.

Simply removing the async keyword from the sleep function fixes this issue.

sleep (timeout) {
  return new Timer(timeout)
}

But, having await in front of a synchronous method seemed a bit odd and unnerving. After a deeper investigation, I found this piece of documentation here:

If the value of the expression following the await operator is not a Promise, it’s converted to a resolved Promise.

In simple terms, calling await on non-promise objects is the same as await Promise.resolve(obj), and it’s a completely expected use case. So while the usage remains a bit uncommon, the code is perfectly valid.

Finally, our http request logic looked like this:

const timer = sleep(timeout)
await Promise.race([
  http.get(‘pocketgems.com/careers/’),
  timer
])
timer.cancel()

Neat!

Are We Done Yet?

Not quite.

The simple timer worked when used exactly as below.

const timer = new Timer(1000)
await Promise.race([timer, anotherPromise])
timer.cancel()

There were other use cases where the Timer class didn’t behave according to intuition:

1. Canceled timer should resolve immediately

const timer = new Timer(1000)
timer.cancel()
await timer // shouldn’t wait for 1s, but it does

2. Timer should start immediately after construction

const timer = new Timer(1000)
const anotherTimer = new Timer(2000)
await anotherTimer // waits for 2s
await timer // shouldn’t wait for 1s, but it does

To explain this behavior, we need to understand that the then method is called only when a thenable object is awaited. In the two use cases above, setTimeout is called on the line await timer. To support these use cases, we needed to call setTimeout when we constructed the timer, not in the then method. The updated code looked like this:

class Timer {
  constructor (millis) {
    this.resolved = false
    this.handle = setTimeout(() => {
      this.handle = undefined
      this.resolved = true
      if (this.resolve) {
        this.resolve()
        this.resolve = undefined
      }
    }, millis)
  }
  
  cancel () {
    if (this.handle) {
      clearTimeout(this.handle)
      this.handle = undefined
      this.resolve = undefined
    }
  }
  then (resolve, reject) {
    if (this.resolved || !this.handle) {
      resolve()
    } else {
      this.resolve = resolve
    }
    this.handle = undefined
  }
}

Now, the timer also works in the “strange” use cases mentioned above.

In Closing

I built the task queue service during a company-wide hackathon, where everyone at PG worked on learning-oriented projects that they wouldn’t have otherwise been able to commit to. In the spirit of hackathon, I spent a bit of extra effort to make this cancelable timer and, in doing so, acquired a deeper understanding of the async and await keywords and thenable objects.

While hackathons don’t happen daily, learning opportunities like this are common in my day-to-day work on project Todea and in many other projects I’ve worked on previously at PG. If you are passionate about building high performance systems and state of the art technologies that enable game developers to deliver high quality games to millions of players, there are profound opportunities for you here.