OpenWhisk deployment on a Kubernetes Cluster

Besides the criticism of Serverless computing [1], it is quite popular these days especially the AWS lambda functions. In this blog, we will deploy OpenWhisk one of the popular opensource serverless computing platforms along with Kubernetes to create a serverless computing environment. Before going into the details of deployment, it is assumed that you have the knowledge of OpenWhisk and Kubernetes architectures.

Kubernetes Cluster Deployment

First, let's deploy the Kubernetes cluster using kubeadm:

1. Install Docker

$ sudo apt-get update
$ sudo apt-get install -y docker.io

2. Install Kubernetes and kubeadm

$ sudo su root
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
$ cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
$ exit
$ sudo apt-get update
$ sudo apt-get install -y kubelet kubeadm kubernetes-cni

3. Initialize and create the Kubernetes Cluster

$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml
$ kubectl taint nodes --all node-role.kubernetes.io/master-

3. Use the join commands on other VMs in the same network to join the cluster

$ kubeadm join 10.128.0.13:6443 --token 9dfcuu.1i81u6xq8bka0hun --discovery-token-ca-cert-hash sha256:850c65aeee8947072ad0905790fe033db4afd8e0263e49382c9a1f5db97a07a7

4. List the nodes part of the cluster using: kubectl get nodes

Configuration steps for Openwhisk

1. Cluster configuration file setup (save as mycluster.yaml):

Here replace the <CLUSTER_IP> with the Kubernetes cluster IP which you can get by running the command (here 10.128.0.13):

kubectl cluster-info

We are creating two replicas of the controller and we are disabling the usage of persistence by adding the following stanza to your mycluster.yaml:

k8s:
  persistence:
    enabled: false

we are using KubernetesContainerFactory, one can read details about them here. Other options are explained in the end when they are used.

2. Installation of the helm for setting up Openwhisk

$ wget https://get.helm.sh/helm-v2.16.1-linux-amd64.tar.gz
$ tar -xvf helm-v2.16.1-linux-amd64.tar.gz
$ sudo mv linux-amd64/helm /usr/local/bin/helm
$ helm init

3. OpenWhisk Command-line Interface (CLI) is a unified tool that provides a consistent interface to interact with OpenWhisk services. Installation of wsk CLI:

$ wget https://github.com/apache/openwhisk-cli/releases/download/latest/OpenWhisk_CLI-latest-linux-386.tgz
$ tar -xvf OpenWhisk_CLI-latest-linux-386.tgz
$ sudo mv wsk /usr/local/bin/wsk

Deployment of OpenWhisk

1. Firstly, we need to specify which of our nodes are core which operates the OpenWhisk control plane (the controller, kafka, zookeeeper, and couchdb pods) and invoker nodes which schedules and executes user containers.

• Label Cores nodes by running this command:

$ kubectl label nodes <CORE_NODE_NAME> openwhisk-role=core

• Label Invoker nodes by running this command:

$ kubectl label nodes <INVOKER_NODE_NAME> openwhisk-role=invoker

• or Label all nodes as part of the cluster as Invoker nodes by running this command:

$ kubectl label nodes --all openwhisk-role=invoker

2. Deploy OpenWhisk using the following commands:

$ kubectl create namespace openwhisk
$ helm init
$ git clone https://github.com/apache/openwhisk-deploy-kube.git
$ cd openwhisk-deploy-kube
$ mv ../mycluster.yaml .
$ helm install ./helm/openwhisk --namespace=openwhisk --name=owdev -f mycluster.yaml

The process will take around 5–10 minutes. This will create all the required Kubernetes components (containers, networks, volumes, etc) required by OpenWhisk. The dashboard of Kubernetes will look something like this (if you have deployed[not covered as part of this tutorial]).

To periodically check the status of the deployment, one can run the following command:

$ helm status owdev

wsk CLI configuration

1. We already have downloaded the wsk CLI to talk to our new cluster — we had downloaded it from here.
2. Configure by setting the auth and API host properties as follows:

$ wsk property set --apihost <master_node_public_ip>:31001
$ wsk property set --auth 23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP

And the token specified is a default token added into OpenWhisk for the guest account.

3. Check if it working or not by listing the installed packages:

$ wsk -i package list /whisk.system

• -i is used because we have not deployed this on secure https cluster hence it allows us to skip the certificate checks.
• The output of the above command:

Sample function deployment

Now our serverless platform is ready for running functions. Let's deploy one simple function following the JavaScript code as:

/**
 * Hello world as an OpenWhisk action.
 */
function main(params) {
    var name = params.name || 'World';
    return {payload:  'Hello, ' + name + '!'};
}

1. Save the code to a file. For example, hello.js.
2. From the OpenWhisk CLI command line, create the action by entering this command:

$ wsk -i action create hello hello.js

3. Invoke the action by entering the following commands.

$ wsk -i action invoke hello --result

• This command outputs: { "payload": "Hello, World!" }

4. Run this command for passing parameter:

$ wsk -i action invoke hello --result --param name Anshul

• This command outputs:{ "payload": "Hello, Anshul!" }

5. Run some invocations and wait for some time(around 15mins) then run again some invocations to check the cold start time.

Monitoring the OpenWhisk deployment

We have installed User-events, Prometheus and Grafana on our cluster with already preconfigured Grafana dashboards for visualizing user-generated metrics using these options in mycluster.yaml file:

- System metrics

For collecting system metrics, store and display them with Prometheus,

metrics:
  prometheusEnabled: true

This will automatically spin up a Prometheus server inside your cluster that will start scraping controller and invoker metrics.

- User metrics

For enabling user metrics

metrics:
  userMetricsEnabled: true

To access the Grafana dashboard get the nodeport by running this command: kubectl get services --all-namespaces and find grafana and its port. The Grafana dashboard can be accessed on: http://<VM_PUBLIC_IP>:<NODE_PORT>/monitoring/dashboards

All dashboards can be viewed anonymously and by default admin Grafana credentials are admin/admin. Use the bellow configuration in mycluster.yaml to change Grafana's admin password:

grafana:
adminPassword: admin

One can see three Grafana dashboard already added as part of it:

Clicking on Action Performance Metrics dashboard one can see the number of function invocations and the number of functions which suffered the cold start:

[1] Hellerstein, J. M., Faleiro, J., Gonzalez, J. E., Schleier-Smith, J., Sreekanti, V., Tumanov, A., & Wu, C. (2018). Serverless computing: One step forward, two steps back. arXiv preprint arXiv:1812.03651.

Please post your thoughts and if something needs to be changed or added please do let me know. You can reach out to me on LinkedIn.

I look forward to your feedback!