Scheduled functions with OpenFaaS

https://github.com/openfaas/faas

OpenFaaS is a popular open source Serverless platform built on Docker. It makes it very easy to build and deploy serverless functions to the web. Once deployed, functions are available via HTTP request. Generally serverless functions are used as part of an event driven architecture, but it can be handy to run a function on a schedule, similar to a cron job.

This post will outline how to setup and deploy a few NodeJS functions using OpenFaaS, as well as schedule them on a regular interval using Redis. Specifically we will be deploying a simple pingfunction to check a websites status/availability and also a score_scraper function that will grab the English Premiership table by scraping it from ESPN.

OpenFaaS can run on a number of providers, Docker Swarm and Kubernetes to name two. This tutorial makes use of Docker Swarm and to get started, head over to the OpenFaaS Github repository and follow these instructions.

Now that OpenFaaS is up and running, install the CLI tool. The CLI will make it extremely easy to build, deploy and test your functions.

# linux
$ curl -sSL https://cli.openfaas.com | sudo sh
# mac
$ brew install faas-cli

Using the CLI, let’s build the ping function’s skeleton. (remaining instructions are for Linux machines)

$ mkdir ping-function 
$ cd ping-function
$ faas-cli new --lang node ping

The CLI has built a few new files and directories. There should be a new sub-directory named ping and it should contain a handler.js file and a package.json file. Open the new ping/handler.js file in your favorite text editor. handler.js is currently exporting a simple function that returns {status: "done"} . Let’s change that to perform an HTTP request to a designated URL and passes back the status code and status message.

// ping/handler.js
"use strict"
const got = require('got')
module.exports = (context, callback) => {
got(context)
.then(res => {
callback(res.statusCode + ' ' + res.statusMessage)
})
.catch(err => callback(err))
}

We’ve written our function, now let’s install our one dependency got in the new ping sub-directory.

$ cd ping
$ npm install --save got
$ cd ..

Our function is now ready to build and deploy.

$ faas-cli build -f ./ping.yml
$ faas-cli deploy -f ./ping.yml

Let’s check to make sure everything is working as expected by passing our function a URL.

  • If deploying to a multi-node Swarm, you’ll need to push the image to your image repository before running deploy. Just make sure you tag the image name appropriately in the ping.yml file i.e. change image: ping to image: <dockerhub username>/ping .
$ faas-cli build -f ./ping.yml
$ faas-cli push -f ./ping.yml
$ faas-cli deploy -f ./ping.yml

Let’s check to make sure everything is working as expected by passing our function a URL.

$ echo https://google.com | faas-cli invoke ping
200 OK

That’s really all there is to building a function and deploying it to an OpenFaaS instance! For more details about the faas-cli tool, checkout the GitHub repository.

Now let’s build something more useful! A function that will scrape the English Premiership standings. We can start out the same way. Make a directory to house our function artifacts.

$ mkdir score_scraper-function
$ cd score_scraper-function
$ faas-cli new --lang node score_scraper

Let’s add our code to the handler…

// score_scraper/handler.js
'use strict';
const cheerio = require('cheerio');
const got = require('got');
const columns = [
'position',
'team',
'played',
'won',
'draw',
'loss',
'for',
'against'
];
const getScores = (context, callback) => {
got('http://www.espnfc.com/english-premier-league/23/table')
.then(html => {
const teams = [];
const $ = cheerio.load(html.body);
$('tbody').children('tr').each((i, elem) => {
teams[i] = $('td', elem);
});

const tableArr = teams.map(elem => {
const stats = [];
elem.each((i, elem) => {
stats[i] = $(elem).text().trim();
});
stats.splice(2, 1);
return stats.slice(0, 8);
});
            const table = tableArr.slice(1, 21);
const tableObj = table.map(arr => {
const teamObj = {};
arr.forEach((item, i) => {
teamObj[columns[i]] = item;
});
return teamObj;
});
callback(JSON.stringify(tableObj));
})
.catch(err => console.log(err));
};
module.exports = getScores

The above code uses got again to make an HTTP request and also uses the handy HTML parsing module cheerio . So just like last time, install those dependencies, build and deploy.

$ cd score_scraper
$ npm install --save got cheerio
$ cd ..
$ faas-cli build -f ./score_scraper.yml
$ faas-cli deploy -f ./score_scraper.yml
$ echo | $ faas-cli invoke score_scraper
[{"position":"1","team":"Manchester City","played":"8","won":"7","draw":"1","loss":"0","for":"29","against":"4"},{"position":"2","team":"Manchester United","played":"8","won":"6","draw":"2","loss":"0","for":"21","against":"2"},{"position":"3","team":"Tottenham Hotspur","played":"8","won":"5","draw":"2","loss":"1","for":"15","against":"5"},
...

Well that’s great! But it’s no fun having to enter CLI commands when I need to check the latest Premiership standings, so let’s use Redis to provide some timed events we can listen to. Redis has the ability to set keys that have an expiration date. When a key expires we can pick up on that event and trigger our function. Using a simple npm module called skej we can write a list of functions we’d like to schedule as well as set them up to be invoked at regular intervals.

skej will take a JSON object containing information about the functions we want to run and start setting expiring keys in Redis that when they expire will trigger our functions. So let’s create the schedule.

$ mkdir skej-template
$ cd skej-template
$ npm init
$ npm install --save skej

Create a new javascript file for our schedule.

// schedule.js
'use strict'
const skej = require('skej')
skej({
single: [
{
name: 'ping', // name of the function on OpenFaaS
data: 'https://google.com' // URL to ping
initialRun: 2, // seconds until first function is run
recurring: 5 // function will run every 5 seconds
onFinished: x => console.log(x.body) // callback
},
{
name: 'score_scraper',
initialRun: 10,
recurring: 60,
onFinished: x => console.log(x.body)
}
],
pipe: []
    /* skej also allows for piped commands, piping the output of one       function into another. Currently, we still need to include an empty array even if not using the piping functionality.
*/
})

Next we need to setup Redis. The fastest way to get up and running is with Docker, and once deployed, we need to enable expiring keys. Use the following to get going.

$ docker run -d -p 6379:6379 --name redis redis
$ docker exec -it redis redis-cli
> config set notify-keyspace-events Ex
OK

We should be all set to run our functions on a schedule.

$ node schedule
starting
scheduled
200 OK
200 OK
200 OK
[{"position":"1","team":"Manchester City","played":"8","won":"7","draw":"1","loss":"0","for":"29","against":"4"},{"position":"2","team":"Manchester United","played":"8","won":"6","draw":"2","loss":"0","for":"21","against":"2"},{"position":"3","team":"Tottenham Hotspur","played":"8","won":"5","draw":"2","loss":"1","for":"15","against":"5"},{"position":"4","team":"Watford","played":"8","won":"4","draw":"3","loss":"1","for":"13","against":"13"},{"position":"5","team":"Chelsea","played":"8","won":"4","draw":"1","loss":"3","for":"13","against":"8"},{"position":"6","team":"Arsenal","played":"8","won":"4","draw":"1","loss":"3","for":"12","against":"10"},{"position":"7","team":"Burnley","played":"8","won":"3","draw":"4","loss":"1","for":"8","against":"6"},{"position":"8","team":"Liverpool","played":"8","won":
....
200 OK
...

And there you have it! Scheduled functions running on OpenFaaS! More information about OpenFaaS can be found here. The skej module is a preliminary proof of concept and certainly a work in progress so please feel free to provide feedback by filing an issue on the GitHub repository.

Comment welcome!