Detect heap overflow on Node.js: JavaScript heap out of memory

TL;DR

Detect “JavaScript heap out of memory” fatal error and avoid node app crash when increasing node’s memory with --max-old-space-size is not an option.

See the code solution on github.

Usecase

Recently I’ve worked on a big data project that involves reading data from neo4j database through some GraphQL API.

Our neo4j database included ~1 Million entries, and I was getting the data from the database with neo4j driver for Node.js.

When querying for results in the GraphQL server, with a limit of 25, all was good, but when I needed to query more that 50 results, the server was dead following JavaScript heap out of memory error — that’s because of the neo4j driver that needed much more RAM than my server application had.

The only solution I could find on google was to increase the memory of my Node.js application via max-old-space-size:

node --max-old-space-size=4096 yourFile.js

But what happens when your machine doesn’t have the required memory?

In my case, I had to work with a machine with a very limited RAM of 512MB.

In this article I’ll show you how to detect/predict situations of “JavaScript heap out of memory” to avoid app crash.

Detect heap overflow

Let’s assume we have some heavy heap consume function:

let heavyHeapConsumer = () => {
    let arrays = [];
    setInterval(() => {
        arrays.push(new Array(1000000));
    }, 100);
};

It just allocates a new array each 100 ms and saves it in some variable named arrays. If you just run it in your node application with heavyHeapConsumer(), it’s just a matter of time until you’ll get a fatal error:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed — JavaScript heap out of memory

This error will terminate your node application, and if your node app is a http server, you’ll stop serving requests.

In order to avoid it, let’s use cluster and v8 core libraries of node.

Cluster allows the user to take advantage of multi-core systems, and launch a cluster of Node.js processes to handle some tasks.

V8 exposes APIs that are specific to the version of V8 built into the Node.js binary.

Now we’ll separate our app to master and worker:

The master will be responsible to restart workers in case they’re dead due a heap overflow, while the worker will be responsible for the main logic (in our case, it just running the heavyHeapConsumer function):

const cluster = require('cluster');
const v8 = require('v8');

let heavyHeapConsumer = () => {
  let arrays = [];
  setInterval(() => {
    arrays.push(new Array(1000000));
  }, 100);
};

if (cluster.isMaster) {
  cluster.fork();
  cluster.on('exit', (deadWorker, code, signal) => {
    // Restart the worker
    let worker = cluster.fork();
    
    // Note the process IDs
    let newPID = worker.process.pid;
    let oldPID = deadWorker.process.pid;
    
    // Log the event
    console.log('worker ' + oldPID + ' died.');
    console.log('worker ' + newPID + ' born.');
  });
} else { // worker
  const initialStats = v8.getHeapStatistics();
  
  const totalHeapSizeThreshold = 
    initialStats.heap_size_limit * 85 / 100;
  console.log("totalHeapSizeThreshold: " + totalHeapSizeThreshold);
  
  let detectHeapOverflow = () => {
    let stats = v8.getHeapStatistics();
    
    console.log("total_heap_size: " + (stats.total_heap_size));
    
    if ((stats.total_heap_size) > totalHeapSizeThreshold) {
      process.exit();
    }
  };
  setInterval(detectHeapOverflow, 1000);
  
  // here goes the main logic
  heavyHeapConsumer();
}

The master code is pretty straightforward: when the app runs on the first time, fork a worker, and bind him a listener for the “exit” event — which will fork a new worker instead, and log this event.

total_heap_size: The size V8 has allocated for the heap. This can grow if used_heap_size needs more.

heap_size_limit: The absolute size limit the heap cannot exceed. e.g.: max_old_space_size

The worker code sets a threshold for total_heap_size which is 85% of the heap_size_limit. Afterwards, the worker periodically checks (in interval of 1 sec) if the limit has been exceeded, and if so — killing himself.

With this approach (worker killing himself when it detects that his heap size is near the max heap size) we can be sure that our node app won’t crash — because the master will assign a new worker to handle the main logic of the app.

Note: you should set the threshold (85%) and the interval (1 sec) accordingly to your app, more specifically — according to the rate of the heap consumption by your main logic (in our case, heavyHeapConsumer was increasing the heap each 100 ms; so if your app increases the heap each 10ms, you should lower the threshold or increase the interval).

The full code can be found on: https://github.com/EvgeniLeonti/detect-heap-overflow-nodejs/blob/master/heapoverflow.js