Serverless & OpenStack

Earlier this month, there was an OpenStack mailing list thread talking about how to fill the new-ish demand for Function-as-a-Service and serverless apps in OpenStack. At the moment, there’s no equivalent in OpenStack that comes close to a Lambda equivalent. So what’s the best way forward if you’re developing on OpenStack?

The Mistral project provides the ability to run workflows in OpenStack, but also requires a DSL for actions and definitely doesn’t support multiple languages. That covers (somewhat) the simple case for Lambda usage, but not the ones that are highest value.

The value of serverless isn’t just being able to hook code up to event sources (S3, Kinesis, etc) and HTTP endpoints (API Gateway), it’s about workflow and developer-friendly deploys. For OpenStack, the real win here would be providing that kind of ease to on-prem or hybrid deployments. Writing a FaaS to be “OpenStacky” probably wouldn’t be the best way to invest contributor time and other resources — after all, if there’s something that works it’s easier to integrate than build.

So what’s out there? IBM’s OpenWhisk project already has an API Gateway, native Swift and NodeJS support, plus the ability to BYO Docker images for any language you like. To fit into the OpenStack Big Tent there would just need to be a few plugins/additions:

1. An auth plugin in OpenWhisk to support Keystone authentication.
2. A prebuilt deployment system to get OpenWhisk going on Nova instances, similar to how Trove and other higher-level projects use Heat to run on the cloud.
3. Base images for use with OpenStack that provide shade and other *-client libraries to make it easy to control OpenStack from OpenWhisk.

In my eyes, that would be enough for a minimum-viable-FaaS on OpenStack. To make FaaS really first-class would take a lot more engineering effort, since an equivalent of DynamoDB isn’t a part of OpenStack. There’s also no Kafka-as-a-Service (Kinesis equivalent).

The hardest thing to add to OpenStack is something I struggled with working on Zaqar, and that’s integrations with event sources in OpenStack. What that’d look like is having Nova, Cinder, Glance, Heat, Trove, and the rest publishing events on user-accessible Zaqar queues. Right now, most services only support events on the internal RabbitMQ message bus, which can’t be exposed to public cloud users.

In large part, that work is on the individual projects to expand API support to configure event queues and publish relevant information for users. The other half of that is cross-project work to find a consistent event format to use with Mistral and OpenWhisk.