OpenWhisk and RabbitMQ
Recently, for an EU H2020 research project (GOFLEX), the opportunity arose to design and build a cloud-native platform to provide time-series services to project partners. And with the increasing number of new services available on the cloud (IBM Cloud), combining several of these into a platform is definitely easier from a provisioning point of view. But challenges remain.
Time-series platforms/applications need to acquire data, and the project’s preference was to submit data to the platform using MQTT. The Watson IoT Platform provided cloud-based MQTT message ingestion, and message queueing was also available via MessageHub. Then we discovered OpenWhisk, which already had a feed mechanism for subscribing to MessageHub topics, and the ability to launch an OpenWhisk action to process the received messages. Great! We now had a way to ingest time-series data onto the cloud platform, through our new OpenWhisk action which stored the data.
Next, retrieving the time-series data in response to queries was necessary, and whilst we can invoke a second OpenWhisk action for data retrieval via the OpenWhisk REST API (with credentials), it was more difficult to integrate this with existing external project partner’s applications. The project already used RabbitMQ for messaging across distributed systems and we wanted to have a standard approach across all services and applications. So ideally, we wanted to publish a request-for-data message to a RabbitMQ queue, and have an OpenWhisk action satisfy this request.
Although a feed mechanism exists to obtain messages from a MessageHub topic and send to an OpenWhisk action, no such capability was available for messages in a RabbitMQ queue. RabbitWhisker was developed to bridge this gap between messages in RabbitMQ queues and OpenWhisk actions, similar to how the MessageHub to OpenWhisk feed works. It subscribes to a RabbitMQ queue, and launches an OpenWhisk action in response to a received message on that queue. It is built as a Docker image, and a container can be launched locally, but for the research project, RabbitWhisker is deployed in a pod on a Kubernetes cluster. RabbitWhisker is described in more detail in this article — “Invoking serverless functions through a message broker”, which contains all the information necessary for deployment, using a cloud based RabbitMQ service.
After extensive use on the project, RabbitWhisker was subsequently open sourced here — https://github.com/ibm-functions/package-rabbitmq.
