Docker vs Functions vs… Browser?

Chris Betts
Nov 4 · 7 min read

… a horribly unfair performance comparison of a number of Google Cloud options vs local computing.

The Mandelbrot Set as a cloud test tool

At last count, there are approximately 1.7 bajillion different options for running a small chunk of code at different levels of abstraction on other people’s computers. You can run code in Docker containers, in Kubernetes, as snippets on GCP Cloud Run or AWS Lambda, as node.js in your local data centre, or even as a javascript on your customer’s web browser.

Which option is best tends to be such a complex tangle of cost, performance, network lag and security posture that it’s almost impossible to give any sensible general guidance, as it will depend on the characteristics of your particular work load.

However on the principle that some data, no matter how dubious, is better than none, here are the results of some messing around I did out of curiosity to try to compare performance of a number of options:

  • parallel code execution with node.js on Google App Engine
  • parallel code execution of Docker containers on Google Cloud Run
  • parallel code execution of raw code on Google Cloud Functions
  • “parallel” code execution of node.js on a local machine
  • single thread raw code in a local browser

The Test Workload — I want All the Pixels!

To exercise the code I chose a ‘pure-compute’ task: calculating, in a 1200x800 pixel grid, the famous Mandelbrot set — using an unoptimised version of the algorithm, this took my code something like 700 million floating point operations. To get some parallelisation I broke the calculation up into six 400x400 blocks, each requiring a bit over 100 million operations.

The goal was to fundamentally get the same code to work across a variety of environments, so I wrote the code in standard javascript, and then deployed it as a single page html file, a node.js file, a docker container wrapping that node.js, and a code snippet for Cloud Functions.

The core code is the two functions below, but each method ‘wrapped’ the code differently (and also handled the UI for the user to select a rectangle to drill down on, because exploring the Mandelbrot set is pretty nifty…!).

function render()
{
for (let x = 0; x < myCanvas.width; x++) {
for (let y = 0; y < myCanvas.height; y++) {
const belongsToSet = checkValue(offsetX + (x / magnificationFactor), offsetY + (y / magnificationFactor));
if (belongsToSet === 0) {
ctx.fillStyle = '#000';
// Draw a black pixel
ctx.fillRect(x,y, 1,1);
} else {
// Draw a colorful pixel
ctx.fillStyle = `hsl(240, 100%, ${belongsToSet}%)`;
ctx.fillRect(x,y, 1,1);
}
}
}
}

function checkValue(real,imaginary) {
let realer = real;
let imaginarier = imaginary;

var real_part, imaginary_part, dist_squared;
const maxIterations = 1000;
for (let i = 0; i < maxIterations; i++) {
real_part = real * real - (imaginary * imaginary);
imaginary_part = 2 * real * imaginary;
dist_squared = real_part * real_part + imaginary_part * imaginary_part;
if (dist_squared > 4) return (i); // return 'closeness'
real = real_part + realer;
imaginary = imaginary_part + imaginarier;
}
// Return zero if (probably) in set
return 0;
}

And the Winner is…

I measured the time taken on Chrome on the network console, which gives a good view of the time taken. We can see the initial html page load, followed by the six connection requests, and then the total load time. (Six connections was chosen as the limit of connections Chrome allows per host — obviously real world applications may benefit from greater parallelisation.) I ran the code a few times to warm up the various platforms, as a cold start for the more heavy weight options such as Docker could be unfair, and almost all real world workloads are against already running systems.

The short version of the results are:

Google Cloud Functions (javascript): 13.03s

local server (node.js): 12.24

Google App Engine (node.js) : 6.09s

Google Cloud Run (docker container): 3.15s

local browser (javascript): 2.72s

So… to be honest I was a bit surprised at this; even taking into account network lag the local browser execution was much faster (and in case you’re wondering, ‘local’ in this case means an antique MacBook that Apple is a little bit embarrassed still exists).

Some bits aren’t surprising — I suspect my local node.js server is not getting any parallelism speedup — but I was surprised at how quick Google Cloud Run was compared to the other options. So for those with a masochistic interest in performance numbers, I’ll run through each test in a bit more detail:

Google Cloud Functions — 13s

The way my code worked, the javascript triggers requests to get the six individual images, and with the Cloud Functions way of doing things we don’t get good visibility of the individual requests… however the end result is pretty lacklustre. Which kind of makes sense; Cloud Functions is designed for relatively light weight use cases largely around co-ordinating other services. I don’t think it’s really meant to be used for number crunching, and I also suspect there are issues around startup overhead.

Local Node.js Server — 12s

Running node.js on my laptop was also fairly uninspiring. It’s not 100% clear to me what’s happening here; index.js gets loaded very quickly and kicks off the six requests. The first request gets dealt with pretty quickly, while the remaining five take disproportionately long to complete. I suspect that my laptop isn’t doing a great job managing multiple execution threads, and I haven’t tried to optimise my local node.js server.

Google App Engine — 6s

So using Google App Engine speeds things up a bit, but despite trying to ‘warm it up’ by calling it multiple times before measuring, it’s still not giving us six responses at roughly the same time. I guess that we’re still probably hitting some sort load balancing / start-up issues. It’s running Docker containers under the cover though, so I expected it to be pretty similar to the results for running Google Cloud Run…

Google Cloud Run — 3s

… however Google Cloud Run actually ran twice as fast. The distribution of calls looks similar though (the first one completes quickly, the rest straggle in later…) so I suspect in both cases there’s a degree of “I’ll run one instance quickly, and spin up others when I need them” — however the claims that Google Cloud Run is better optimised and running on more recent software seem to be correct, it’s significantly faster.

Local Browser — 2.7s

Execution on the local browser is a little more difficult to evaluate (as the javascript only starts running after the page html loads), however as a workaround it seems that the ‘favicon’ load pauses until after the main page has rendered. (I confirmed this independently by adding an internal timer in the rendering code.)

The results above show the single threaded browser code as the clear winner! Even taking into account network lag, it is still significantly faster than any of the other options.

What’s it all mean?

I was actually surprised at all of this — I thought that running six loads in parallel on the cloud would give a significant speed up — but I think really all it’s saying is that container management is hard work, while simple computation is pretty cheap (800 mega-flops on my old 2.2GHz i7 laptop should still execute in around a second or so).

I think what I’ve learnt here is that there is a bunch of overhead in managing these containers (even Google Cloud Functions is running on some sort of management substrate), and my toy workload probably isn’t exercising these systems conventionally enough to have them running enough instances to quickly respond. I’m guessing there’s a lot of whirring and grinding going on in the background every time I hit refresh as containers pop up and down.

So can I use these numbers for my business case?

A whole world of ‘no’!

Real use cases will have far more complex networking requirements, will need to access data stores, communicate with other processes, and will likely have far heavier load. Under those conditions there is still no substitute for performance testing with a load close to what your real load will be.

Also, I should emphasise that I made no effort at all to optimise these systems for my particular use case — this was simply botched together using whatever the default environment was out-of-the-box…

However, what this does show is that it may not be too difficult to write your code in such a way that you can try it out in multiple environments— and that there are significant differences between execution environments running the same code.

So break out your container code and try out all the different mechanisms and vendors on offer — there are some big improvements in performance and costs available for remarkably little effort.

And don’t forget that running your code on your customers’ browsers may occasionally be the cheapest and fastest option of all…!

Chris Betts

Written by

Cloud and Identity / Security SME living in Melbourne, and enjoying messing around with everything from airships to blockchain to cloud…

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade