[Ballerina] Event Notification via Webhooks

Webhooks facilitate a push-based content delivery/notification mechanism between sources of events and interested parties.

The operation of a webhook would thus constitute of:

1. An interested party (subscriber) registering an HTTP callback, at a source, to receive notification on occurrence of a particular event
2. The source notifying the occurrence of the event, via an HTTP POST request to the registered callback URL

Popular service providers allowing webhook notifications include GitHub, stripe, SendGrid, etc.

While this seems pretty straightforward, things get a bit complicated when you consider multiple subscribers registering callbacks for multiple events, managing all these subscriptions, securing webhooks, etc.

Enter WebSub, previously PubSubHubbub, which introduces a communication mechanism between publishers of content and subscribers interested in the content via intermediary hubs, based on HTTP Webhooks.

The basic flow with WebSub is as follows:

1. A subscriber discovers from a publisher, the topic it needs to subscribe to and a hub it needs to subscribe at, to receive notifications for a particular event
2. The subscriber sends a subscription request to a hub, specifying the discovered topic along with the callback URL at which notification is expected. This request could also include a lease seconds value indicating how long the subscriber wants the subscription to be active and a secret to use for authenticated content distribution
3. The hub sends an intent verification request to the callback URL, and if verified, the subscription is added
4. Once an event occurs, the publisher notifies the hub of updates to the topic
5. The hub delivers the identified updates to the subscribers of the topic

Ballerina

Ballerina is this cool new programming language, optimized for integration (Disclaimer — might be a bit biased here ;)). Ballerina comes with built-in support to address a wide range of integration needs. Do checkout the official resources for more information on the available features and to set up and try out Ballerina!

Event Notification with Ballerina Webhooks

Ballerina has built-in support for Webhook based event notification, with its implementation of the WebSub specification. You can find the API docs for ballerina-websub at https://ballerina.io/learn/api-docs/ballerina/websub.html, and basic Ballerina WebSub examples under the WebSub section of https://ballerina.io/learn/by-example/.

In this post we will be looking at a simple scenario to understand how we can use Ballerina’s WebSub implementation to integrate event notification.

Consider a simpler version of the order management service specified in the RESTful service Ballerina guide, which just accepts orders.

Publisher and Hub

To demonstrate webhooks, let’s assume that we are interested in receiving a notification every time someone “places an order” (i.e., every time http://localhost:9090/ordermgt/order receives a request).

The function startHubAndRegisterTopic(), which starts up a Hub and registers the topic against which updates will be published, is invoked and the returned websub:WebSubHub object is assigned to a global variable which will be used within the addOrder resource to publish updates.

Ballerina’s WebSub Hub implementation would then handle all subscription acceptance, management, notification, etc.

Subscriber

Let’s now set up a Ballerina WebSub Subscriber Service which would accept event notification for orders placed.

Subscribing to receive notifications on event occurrence becomes pretty straightforward via Ballerina’s subscriber service endpoint and service which allow specifying required configs/parameters as endpoint configurations and/or annotations.

Note how we have specified all the subscription details including the hub to subscribe at and the topic to subscribe to along with a lease seconds value and a secret as annotations.

The single resource, onNotification, is the resource to which all content delivery requests would be dispatched.

With this service, on start up, a subscription request would be sent to the hub http://localhost:9191/websub/hubfor the topic http://localhost:9090/ordermgt/ordertopic specifying the lease seconds and secret values, registering the callback http://localhost:8181/ordersubscriber.

When an intent verification request is received as a result, intent to subscribe would be verified if the topic specified in the intent verification request matches that annotated.

Intent Verification and Signature Validation

With this service, intent verification for subscription/unsubscription requests happens automatically. Explicit intent verification could be done by introducing an onIntentVerification resource and introducing custom logic to accept/deny verification.

Since a secret is specified for the subscriber service, every content delivery request would have to contain an X-Hub-Signature header as per the spec, and signature validation would be done automatically to validate the header value. Only requests for which signature validation is successful would be dispatched to the onNotification resource.

Demo Time!

Let’s now take a look at how all this fits together.

First, start up the order management service along with the hub. Navigate to the folder containing order_mgmt_service.bal and run,

$ ballerina run order_mgmt_service.bal
ballerina: initiating service(s) in 'order_mgt_service.bal'
ballerina: started HTTPS/WSS endpoint 0.0.0.0:9191
ballerina: Default Ballerina WebSub Hub started up at https://localhost:9191/websub/hub
ballerina: started HTTP/WS endpoint 0.0.0.0:9090

Note how the WebSubHub is started up at https://localhost:9191/websub/hub.

Then start up the subscriber service:

$ ballerina run subscriber_service.bal
ballerina: initiating service(s) in 'subscriber_service.bal'
ballerina: started HTTP/WS endpoint 0.0.0.0:8181
2018-07-21 16:56:29,784 INFO  [ballerina/websub] - Subscription Request successful at Hub[https://localhost:9191/websub/hub], for Topic[http://localhost:9090/ordermgt/ordertopic], with Callback [http://localhost:8181/ordersubscriber] 
ballerina: Intent Verification agreed - Mode [subscribe], Topic [http://localhost:9090/ordermgt/ordertopic], Lease Seconds [36000]

Note how a subscription request is sent to the specified hub with the subscription parameters we specified, for the callback corresponding to the subscriber service. Intent to subscribe is also verified once an intent verification request is received.

We can now take a look at the event notification, by invoking the order management service to add orders with different order ids.

For example, the following two requests would result in adding orders with order ids 100500 and 110500:

$ curl -X POST -d '{ "Order": { "ID": "100500", "Name": "XYZ", "Description": "Sample order."}}' "http://localhost:9090/ordermgt/order" -H "Content-Type:application/json"
{"status":"Order Created.","orderId":"100500"}
$ curl -X POST -d '{ "Order": { "ID": "110500", "Name": "XYZ", "Description": "Sample order."}}' "http://localhost:9090/ordermgt/order" -H "Content-Type:application/json"
{"status":"Order Created.","orderId":"110500"}

These order should result in the subscriber service receiving notifications, resulting in logs similar to the following:

2018-07-21 17:17:52,513 INFO  [] - WebSub Notification Received: New Order Added: 100500 
2018-07-21 17:17:59,780 INFO  [] - WebSub Notification Received: New Order Added: 110500

There, we have a simple demo of setting up event notification via webhooks with Ballerina.

To check out how you can integrate a Ballerina WebSub Subscriber Service with an external service provider allowing webhook registration, check out https://medium.com/@maryamzi/tweet-my-stars-ballerina-3a9652226116, which demonstrates a GitHub webhook.

Hope you found this post useful! :) Do stay tuned for more posts looking into each WebSub component in detail.

Thanks!