During my time at iHeart I did get my chance to write an image manipulation micro-service that deals with the myriad of image sizes that iHeart’s supported devices use. It was definitely a challenging experience that still gives me performance-anxiety till this day. A lot of technical decisions were made and for various reasons I decided to get my hands dirty on node.js + GraphicsMagick.
Scouring through the list of npm modules that supports either ImageMagick (IM) or GraphicsMagick (GM) there seems to (intuitively) be 2 types of drivers: native & non-native. Since node.js supports process I/O via stdout, stdin & stderr, the most popular driver (gm) seems to use this approach, since it’s a lot easier to write a driver that abstracts the CLI of GM/IM, but at the cost of performance of course (serializing and piping binary data to stdin, spawn convert process, then pipe from stdout to response stream.
The other alternative was node-imagemagick which unfortunately is not compatible with GM (IM vs GM is another discussion). The other thing I’m not a big fan of this driver is its interface. The CLI for GM/IM allows you to essentially chain operations infinitely and re-mapping it to certain operations removes that chainability (I can’t resize, then blur, then resize, then crop, arbitrarily). There’re also a couple of perfomance tricks that IM suggests that requires chainability (blurring a huge image is a lot faster if you downsize it, the blur, the scale it up).
With that said, I decided to write my own sauce node-gm-native that offers an interface very similar to GM’s CLI. It’s still in its early phase but essentially the big difference is that it offers chainability.
There was unfortunately not a whole lot of docs on writing native Node modules, but nan is a very good start. It has a list of common use cases and examples and do abstract out a lot of V8's data types for you. It’s incredibly helpful since going from JS to C++ (dynamic typing to static typing), the thing I was struggling w/ the most is actually type conversion. V8 fortunately is OO so you can easily look up its API and sometimes source code, then from there figure out conversion from V8 types to std types, or C types. You can get started w/ writing a simple async worker here (Async Worker Example)
The driver itself is fairly straightforward and just a matter of operation string -> operation method calls, although there were some key operations missing from GraphicsMagick++ itself that I had to patch myself (P.S: GM uses Mercurial but uses SourceForge to keep track of its issues, which is fairly hard to work with). Cross-platform support is mostly supported out of the box, w/ certain libs path having to be manually configured dependent on OS and sometimes OS version (CentOS, OSX 10.9 vs 10.10). The hard part is really benchmarking and profiling. Although V8 as a VM does provide GC, there were definitely issues going from JS SlowBuffer land to C++ buffer land and is susceptible to memory leaks.
All in all, it came down to the benchmark:
86ops/s vs 40.18ops/s and since it’s native, it scales by the # of cores (Spawning is susceptible to OS-dependent zombie-process cleanup & file-descriptor management).
