Fn Hot Docker Functions
Ideally, an Fn function should be “hot”. With a hot function, the function container is started and kept alive to process a series of function invocations. Hot functions are great because there’s a cost to starting a container — the Fn server has to pull the image from a repository if the image isn’t cached locally, start the container, and typically boot the language runtime inside the container before it can even start handling function calls. With a hot function you only pay these “cold start” pull/start/boot costs on the first invocation. Latency on subsequent invocations is dramatically lower. So naturally you should always deploy a hot function, right? Unfortunately, it’s not so straight forward if you’re deploying a Docker container as a function — but there is a way!
If you’re using any of the Fn Function Development Kits (FDKs) hot functions are a breeze. In fact, you are probably using an FDK and didn’t really think about whether your function was hot or not. You included the FDK library in your application, wrote your handler function (or method), and passed your handler to the FDK. The fact that the container is kept alive and that the FDK is providing a request loop listening for incoming function invocations may not be obvious. But that’s what’s happening.
If you aren’t using an FDK then you’re running your function with the “default” (i.e., “not hot”) container contract. With default, a container is started for each function invocation, input is piped into the container on stdin, the response read from stdout, and then it’s shutdown. It works but doesn’t provide the low latency response time of a hot function.
Hot Docker
In addition to support for popular languages through FDKs, Fn supports deployment of Docker containers as functions. This is a major differentiator when compared to many other functions-as-a-service platforms. Fn lets you deploy anything as a function as long as your container implements one of the supported container contracts. Fortunately, the default stdin/stdout container contract is easy to implement in your custom containerized function. For example, here’s a Dockerfile that packages up a Node.js program as a container image that can be deployed to Fn as a function.
NOTE: This example is just for illustration purposes and you should really be using the Fn Node FDK for optimal Node.js performance!
And here’s thefunc.yaml file that allow you to build this function with fn build. Notice that the (container contract)format is default and runtime is set to docker rather than java, node or one of the other languages supported by Fn.
So with these three files you can deploy a default container contract function. But this function won’t have the low latency it would have if it were a hot function. On each invocation you will pay the container and Node.js runtime startup costs. If I deploy to a local Fn server and invoke this function with time I see a response time of around 1.3 seconds. That’s pretty bad, but not an entirely surprising amount of time to start a Docker container on a Macbook.
“Hotwrap” Prototype
It would be much better if I could run this code as a hot function. Fortunately, to make it easier to deploy non-FDK functions as hot functions a prototype utility with codename “hotwrap” is currently being developed. Hotwrap implements the hot function container contract and executes your stdio-based function each time the function is invoked. Using hotwrap, the function container is kept alive so you avoid container startup costs — although you still pay the cost to start your function executable on each invocation.
To use hotwrap you simply set the hotwrap executable as the ENTRYPOINT of your container and provide theCMD to launch your function executable. The Dockerfile below illustrates the changes required to add hotwrap to the Node.js example above.
NOTE: This example uses a multi-stage Docker build to obtain the hotwrap executable from a Docker image which currently is not yet published in Docker Hub. You can build this image yourself from the sources on GitHub.
With this updated Dockerfile we can redeploy the Node.js function and take some new timings. On the first invocation, time reports 0.775 seconds to execute the function. This elapsed time includes the cost of starting the container and Node.js. For subsequent invocations, time reports around 0.2 seconds which is a significant 0.5 seconds faster! Thanks to hotwrap we’re only paying the cost of starting the Node.js runtime and running our code.
The future of Hotwrap
Hotwrap is a prototype so don’t use it in production! But it has demonstrated the feasibility and value of providing a mechanism in Fn that improves the runtime performance of programs packaged as functions that were not originally written as functions. While Fn has always supported the deployment of Docker containers as functions, those functions used the default container contract and incurred higher latency costs. Hotwrap partially addressed the disparity so look for an official (likely renamed) hotwrap utility in Fn in the near future!
Thanks to Owen Cliffe for developing hotwrap!!
