OpenWhisk and RabbitMQ

Mark Purcell
Sep 13, 2019 · 2 min read

Recently, for an EU H2020 research project (GOFLEX), the opportunity arose to design and build a cloud-native platform to provide time-series services to project partners. And with the increasing number of new services available on the cloud (IBM Cloud), combining several of these into a platform is definitely easier from a provisioning point of view. But challenges remain.

Time-series platforms/applications need to acquire data, and the project’s preference was to submit data to the platform using MQTT. The Watson IoT Platform provided cloud-based MQTT message ingestion, and message queueing was also available via MessageHub. Then we discovered OpenWhisk, which already had a feed mechanism for subscribing to MessageHub topics, and the ability to launch an OpenWhisk action to process the received messages. Great! We now had a way to ingest time-series data onto the cloud platform, through our new OpenWhisk action which stored the data.

Data Acquisition using OpenWhisk actions

Next, retrieving the time-series data in response to queries was necessary, and whilst we can invoke a second OpenWhisk action for data retrieval via the OpenWhisk REST API (with credentials), it was more difficult to integrate this with existing external project partner’s applications. The project already used RabbitMQ for messaging across distributed systems and we wanted to have a standard approach across all services and applications. So ideally, we wanted to publish a request-for-data message to a RabbitMQ queue, and have an OpenWhisk action satisfy this request.

Data Retrieval using OpenWhisk actions

Although a feed mechanism exists to obtain messages from a MessageHub topic and send to an OpenWhisk action, no such capability was available for messages in a RabbitMQ queue. RabbitWhisker was developed to bridge this gap between messages in RabbitMQ queues and OpenWhisk actions, similar to how the MessageHub to OpenWhisk feed works. It subscribes to a RabbitMQ queue, and launches an OpenWhisk action in response to a received message on that queue. It is built as a Docker image, and a container can be launched locally, but for the research project, RabbitWhisker is deployed in a pod on a Kubernetes cluster. RabbitWhisker is described in more detail in this article — “Invoking serverless functions through a message broker”, which contains all the information necessary for deployment, using a cloud based RabbitMQ service.

After extensive use on the project, RabbitWhisker was subsequently open sourced here —

Apache OpenWhisk

Apache OpenWhisk is a serverless cloud platform that executes code in response to events

Mark Purcell

Written by

Technical Architect at IBM Research — Ireland

Apache OpenWhisk

Apache OpenWhisk is a serverless cloud platform that executes code in response to events

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade