NodeJs internals: V8 & garbage collector
In this article we will see :
- why a garbage collector is important!
- V8’s garbage collector algorithms (Scavenge, Mark/sweep/compact)
- V8’s new GC optimizations (concurrent, parallel, …)
Why a garbage collector?
In the past when you were writing a program you would do strange things like malloc or free to manage your program’s memory.
In modern languages, the Garbage Collector (GC) handles all of this for you. The same is true for NodeJS which uses V8 to GC and execute all javascript code.
A clear understanding of the mechanisms behind GC can help you avoid the ultimate developer’s nightmare; memory leaks. Furthermore, GC monitoring can help you optimize your application.
Why should you take care of GC metrics?
I work at Voodoo, a French company that creates mobile video games. We have a lot of problems with performance, availability, and scalability because of the insane amount of traffic our infrastructure supports (billions of events/requests per day …… no joke!). In this setting, every aspect of web development is important and every decision can have a major impact on our business.
Most of the time, when working on a low traffic API or project we don’t realize how many tweaks or improvements we can do. On the other hand, when traffic increases, all previous oversights can cost dearly.
In 2009, Amazon found that every 100ms of latency cost them 1% in sales.
So a sound comprehension of GC’s mechanisms with good monitoring of its metrics can help you build a more robust application, especially with high traffic level projects.
GC algorithm
A garbage collector does not necessarily clean up memory the same way each time. With V8 there are 2 main algorithms: Scavenge and Mark/Sweep/Compact.
Scavenge
This algorithm can run many times and cleans a small amount of memory. When monitoring GC cycles you will probably see the scavenge algorithm run the most.
Minor GC algorithm. It’s used only in new space.
As shown in the picture above a scavenge cycle basically copies memory in a “to” space into “from” space, then moves all “alive” objects back into “to” space. If some objects are too old they are moved to “old space”.
When the algorithm ends, “to space” only contains alive objects that should be kept. This kind of GC cycle has important limitations inherent in the algorithm: moving objects into memory. This is why it’s only used on new space for frequent (but small) memory cleaning.
Mark / Sweep / Compact
This algorithm is based on a simple approach: mark objects in memory to know which ones are still “alive”, then sweep the memory to clean “dead” (unused) objects. And finally, compact memory to optimize it.
Mark phase:
Recursive procedure of marking reachable objects !
V8 uses the white/gray/black marking system.
Basically, this marking system can be summed up like this:
- white: the initial state, this object has not yet been discovered
- gray: object has been discovered
- black: object and all of its neighbors discovered
To better understand this phase, it is helpful to see memory as a tree. Each node represents an object in memory and each branch represents a reference between objects.
In the beginning, all “root” nodes will be automatically flagged as gray because the GC already knows about them. In the NodeJs world, a root node is typically the object “global” or “process”.
When the GC has discovered all nodes they will all be marked in black …… or not. All remaining “white” objects will be removed during the sweep phase.
Sweep phase
Remove all unused (white) objects.
The entire purpose of the previous marking phase is to mark objects whose memory can be released in this phase.
Compact phase
Moves all marked — and thus alive — objects to the beginning of the memory region.
Another optimization of memory! The GC does not only remove unused objects but also tries to keep memory organized to allow for faster memory reads.
After the sweep phase, the freed memory can be separated by blocks of memory that are still in use.
What’s new in Onirocco?
Cleaning and compacting memory are not the only improvements and optimizations that V8 can do for you. The initial mechanism behind GC cycles was the following: your program is stopped so GC can do its job a.k.a “stop the world” syndrome.
So here’s how the V8 team handled it.
First, introduce “incremental marking”
“Mark” phase takes too much time? Then split it!
Garbage collector splits up the marking work into smaller chunks
Of course, the total time of all GC cycle is still the same, but the main thread is available more often to execute your code. It gives a feeling of increased fluidity in the execution of your program.
Lazy sweeping
And what about the sweep phase? ….. well, it’s the same, but it’s called lazy sweeping!
Sweep pages on an as-needed basis until all pages have been swept
The sweep phase can be done immediately … or it can be delayed. If the amount of free memory is enough to execute your code, why execute a full sweep phase immediately?
It can also be done in parallel. Let’s see how!
Concurrent
The latest versions of V8 are able to execute GC cycles on many different threads. You can benefit from the split mechanism but with a parallel approach, reducing the global GC time.
But you still see a kind of “stop the world” principle here. Is there any way to get rid of it? Of course! And it’s called “parallel”.
Parallel
When possible, V8 is able to perform GC operations in parallel with your code.
What V8 can do? All of this :)
In reality, V8 chooses the best way to manage your memory and perform GC operations. It can apply all of the previous mechanisms, all together, as the image below sums up.
How to monitor memory & GC?
V8 module
The v8 module exposes APIs that are specific to the version of V8 built into the Node.js binary
There are 2 important methods in this module :
- v8.getHeapStatistics() : global memory overview
- v8.getHeapSpaceStatistics() : memory split by space (new, old, …)
Basic output:
gc-stats module
Exposes stats about V8 GC after it has been executed.
First of all, this module is not a built-in module in nodejs. So you need to install it.
npm install gc-statsThis module allows you to listen to the special event “stats” to retrieve GC cycles data.
V8 options
V8 is able to print some valuable information about its state and especially about how the garbage collector is working. In order to display this additional data, you must pass to V8 (so to NodeJs) arguments before starting your application.
node --trace_gc app.jsOne of the most interesting options for getting some GC data is --trace_gc. A typical output should look like this:
Finally, you can find a full list of V8 options here:
https://gist.github.com/listochkin/10973974APM
Last but not least: never forget to use an apm (Application Performance Management) to track your memory usage in real time.
Some of them are also able to display a few metrics about the garbage collector, see image below.
Unfortunately, the current state of art for the monitoring world is not really up to date with the NodeJs garbage collector. Indeed, after testing a lot of apm tools, I found that most of them don’t display all available data and that the majority do not even have this feature.
Conclusion
Understanding the garbage collector and why its role is central in your program memory management is crucial if you want to master what NodeJs and V8 can do for you. Monitoring and debugging the GC’s cycles and memory state can help you find some performance bottlenecks and prevent server crashes.
It’s also good practice to monitor and set up some automatic alarms on your GC’s metrics, like the average GC cycle duration (which should not exceed a few milliseconds) or the frequency of GC’s pause.
You can find my original talk here: https://docs.google.com/presentation/d/17PXOZXBgdJHqBZlbULHg70_hVFLrNx9T34rB9bLDn2M/edit#slide=id.g3ffed3592f_0_0
Many thanks to Kyle Cheng for correcting my English and my awkward turns of phrase.
