Deploying OpenWhisk on Kubernetes

Priti Desai
Jun 20, 2018 · 4 min read

I have been running an experiment from past couple of days to deploy OpenWhisk on Kubernetes to create an easy and quick playground. I am newbie 👶 to Kubernetes and I guess that’s the reason I took one and half day to have a successful deployment otherwise GitHub repo incubator-openwhisk-deploy-kube has very clear instructions on how to deploy OpenWhisk on Kubernetes.

Here is the list of instructions I followed on my Mac, but before you start experimenting, please note that:

OpenWhisk deployment only and only works with Kubernetes 1.8.0 or 1.9.0 and minikube 0.25.2. I experimented using Kubernetes 1.9.0 with single node cluster.

  1. Install asdf package manager and edit ~/.bash_profile or equivalent:
$ brew install asdf
# add the following statement in ~/.bash_profile
[ -s "/usr/local/opt/asdf/asdf.sh" ] && . /usr/local/opt/asdf/asdf.sh
$ source ~/.bash_profile

2. Add minikube and kubectl plugins:

$ asdf plugin-add kubectl
$ asdf plugin-add minikube

3. Install minikube and kubectl using asdf :

$ asdf install kubectl 1.9.0
$ asdf global kubectl 1.9.0
$ asdf install minikube 0.25.2
$ asdf global minikube 0.25.2

4. Create the minikube VM:

$ minikube start --cpus 2 --memory 4096 --kubernetes-version=v1.9.0 --extra-config=apiserver.Authorization.Mode=RBAC

5. Setup Docker network in promiscuous mode:

$ minikube ssh -- sudo ip link set docker0 promisc on

So far, I have kubectl and minikube:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2017-12-15T21:07:38Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"", Minor:"", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2018-01-26T19:04:38Z", GoVersion:"go1.9.1", Compiler:"gc", Platform:"linux/amd64"}
$ minikube version
minikube version: v0.25.2

6. Create namespace OpenWhisk and label Kubernetes worker nodes:

$ kubectl create namespace openwhisk
$ kubectl label nodes --all openwhisk-role=invoker

7. Clone GitHub repo and define the type of ingress in yaml file:

$ git clone https://github.com/apache/incubator-openwhisk-deploy-kube.git
$ cd incubator-openwhisk-deploy-kube/helm
$ vim mycluster.yaml
whisk:
ingress:
type: NodePort
api_host_name: 192.168.99.100
api_host_port: 31001
nginx:
httpsNodePort: 31001

8. Deploy with Helm:

$ brew install kubernetes-helm
$ helm init # init Helm Tiller
$ kubectl get pods -n kube-system # verify that tiller-deploy is in the running state
$ kubectl create clusterrolebinding tiller-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
$ helm install . --namespace=openwhisk -f mycluster.yaml

9. Verify that all pods are in running state:

$ kubectl get pods -n openwhisk
NAME READY STATUS RESTARTS AGE
apigateway-7dbf56f8b-ppqjf 1/1 Running 0 55m
controller-0 1/1 Running 4 55m
couchdb-674b4dbd8d-q22lc 1/1 Running 0 55m
invoker-cl6js 1/1 Running 0 55m
kafka-0 1/1 Running 0 55m
nginx-6d66cccd6-qxgns 1/1 Running 0 55m
redis-7d4dbd56c7-qnzbq 1/1 Running 0 55m
zookeeper-66774495d6-m8phc 1/1 Running 0 55m

Hooray! 💃 We have a working instance of OpenWhisk on Kubernetes cluster. We can start playing with our OpenWhisk instance using wsk CLI or wskdeploy, after configuring host and auth with:

$ wsk property set --apihost 192.168.99.100:31001
$ wsk property set --auth `cat ../kubernetes/cluster-setup/auth.guest`
$ wsk -i list
Entities in namespace: default
packages
actions
triggers
rules

Troubleshooting:

I ended up spending more than a day troubleshooting and debugging OpenWhisk deployment on Kubernetes due to following issues:

After carefully following through the instructions above, while verifying that OpenWhisk, I was bummed to see getsockopt error.

$ wsk -i list
error: Unable to obtain the list of entities for namespace 'default': Get http://192.168.99.100:31001/api/v1/namespaces/_/actions?limit=0&skip=0: dial tcp 192.168.99.100:31001: getsockopt: connection refused

After hitting this error, I checked the state of pods under openwhisk namespace with kubectl get pods -n openwhisk and found one faulty with CreateContainerConfigError.

Some versions of Kubernetes does not work for OpenWhisk deployment due to bug with volume mount subpaths (see issue #61076). Ngnix configuration has configMap volume in ngnix.yaml. The fix is to pick a version of Kubernetes which doesn’t have this bug, like 1.9.0. Now, my concern was I had made sure to install 1.9.0 version of Kubernetes than why am I running into this kind of issue. I tried to verify and found that my server was running 1.9.4 😏

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2017-12-15T21:07:38Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"", Minor:"", GitVersion:"v1.9.4", GitCommit:"bee2d1505c4fe820744d26d41ecd3fdd4a3d6546", GitTreeState:"clean", BuildDate:"2018-03-21T21:48:36Z", GoVersion:"go1.9.1", Compiler:"gc", Platform:"linux/amd64"}

The reason was I started minikube without any parameters and by default it picked up 1.9.4 😱

minikube start

v/s

minikube start --cpus 2 --memory 4096 --kubernetes-version=v1.9.0 --extra-config=apiserver.Authorization.Mode=RBAC

Invoker might take long time to pull all possible runtimes specified in runtimes.json. If it takes too long and if you would like to have OpenWhisk running quickly, you can pull minimal runtimes by adding runtimes-minimal-travis.json to mycluster.yaml:

whisk:
runtimes: "runtimes-minimal-travis.json"

Please feel free to give it a try and let me know if you have any questions or run into any issues. Good Luck 👍

Apache OpenWhisk

Apache OpenWhisk is a serverless cloud platform that…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store