A Serverless Composition of Functions

Connecting Travis CI to Slack Using Apache OpenWhisk

In this article, I show you how to use Apache OpenWhisk to create a serverless application that is composed from serverless functions. The functions are created and deployed as OpenWhisk actions. Similarly, the composition itself is an OpenWhisk action. This is particularly noteworthy in that compositions, therefore, may be further composed, leading one to develop reusable and modular serverless function-libraries.

The serverless application that I describe here is a Travis CI-to-Slack bot. It reacts to Travis CI build notifications and performs the following tasks:

1. retrieves the pull request and build details,
2. fetches and analyzes the build and test logs,
3. generates Slack notifications for the failed tests aimed at contributors.

In the case of a build failure, the GitHub contributor receives a summary of the failing tests as a result of their pull request, along with links to access the complete Travis CI logs. The bot may be configured to also generate a notification for successful builds, which is useful to nudge committers to review a pull request.

Figure 1: Example Slack notifications for Pull Requests submitted to Apache OpenWhisk and tested with Travis CI. The message provides links to the pull request, the build logs, and the test logs.

The source code for the Travis-to-Slack bot is available on GitHub. I credit my former IBM Research colleagues Philippe Suter, for developing an earlier version of this application, and Nick Mitchell, for helping to realize the composition in its current form.

The application control and data flow are illustrated in the figure below and consists of the following logical steps, each of which is a serverless function.

Figure 2: The steps that make up the application.

• extract: The build notification contains several properties to extract, including the repository, pull request number, author name, and build details.
• fetch job id: queries Travis CI to determine the job id. This id determines the location of the build logs.
• verify log url: determines the location for the Travis CI logs from the job id. Travis may store the logs internally or archive them on Amazon’s S3. There is a short window where neither location is valid. This action determines which location to use and fails if there are no logs yet (in which case, the function should be retried).
• analyze log: fetches the logs from the given URL and analyzes them for test failures.
• format message: this action combines the results of the log analysis with the build information extracted earlier to generate the contents of the notification message.
• post notification: sends the Slack message to the intended recipient. This action requires a Slack bot token.

The functions are deployed as OpenWhisk actions and comprise the building blocks from which the application’s control and dataflow are orchestrated to match the architecture shown in Figure 2. The easiest way to do this is using a function combinator library called composer available as an npm package. It permits a hierarchical composition of functions and provides the mechanisms to introduce out-of-band dataflows necessary in my scenario.

Here is the Travis CI to Slack notifier using composer. It makes it easy to write code that matches the intended architecture.

Figure 3: The application composition logic, authored using the Cloud Shell. Composer is a Node.js library. You can reference serverless functions by name, or write inline JavaScript functions.

The top level combinator (line 7) is a sequence. It chains functions together so that the result of one function is the input to the next. The composition can refer to functions by name (e.g., extract) or inline JavaScript functions (lines 14 and 18).

The retain combinator (line 10) is an example of program state that is automatically saved and restored around functions or other combinators. In this case, the extracted build metadata which is the result of the computation after fetch job id is saved automatically on a stack, and restored later after the log analysis completes. In this way, data is persisted and forwarded around other functions.

The retry combinator is needed around the verify log url action since it might require one or more tries to determine the location of the logs to analyze.

The retain on line 16 permits the injection of a Slack token into the formatted notification message. This eschews the need to create a Slack package binding while still using the OpenWhisk package catalog of handy functions. This is an example of a 3rd party function that I didn’t have to write and instead just pulled from a public functions catalog.

The action format message must perform a mapping from the GitHub user id to the contributor’s corresponding Slack id. This function currently assumes the existence of a fixed mapping of names to ids. This is not strictly necessary as the Travis build notification includes the author’s email address. So it is plausible to generate an email notification if no Slack id is found. This could be done using a try-catch or if-then-else combinator for example, and left as an exercise for the interested reader (I will accept pull requests that improve and enhance the composition).

A script is provided to deploy the actions and the composition. Once deployed the OpenWhisk CLI or API may be used to invoke the latter as any other action. In fact, I externalized the application as a web action and used the resultant API endpoint in the Travis configuration to generate the webhook.

That’s it. I hope it’s evident in this post how composer helps you write serverless applications by describing complex control and dataflow between simpler functions.

This application is up and running using IBM Cloud Functions for the benefit of the growing set of OpenWhisk contributors on our Slack channel. To lean more about OpenWhisk, I suggest my recent article on the state of the project.