Running Cellery on top of K3S

In this article, I am going to explain how you can deploy and run the Cellery on top of K3s. Before I get into the installation steps in detail, I think it is worth to describe briefly what is Cellery and K3s, and how that will help to solve your problems with kubernetes and micro services architecture.

What is Cellery?

With the increased use of microservices and serverless applications within organizations, the situation becomes out of control within a few months or years with few to hundred thousands of such applications that interact among them. Hence the development of new features, updating them without affecting other applications/teams and managing them becomes a nightmare as there is no clear boundary, API, and connections between those applications. This ultimately affects the developer workflow and agile development.

This brings the necessity of more high-level construct that aggregates microservices and serverless functions together as a single reusable building block with a clear well-defined APIs. This is known as Cells. Cellery is a code-first approach to building, integrating, running and managing composite applications (cells) on Kubernetes. It allows to build, push/pull, test, deploy, update, scale and observe cells. APIs of the cells are secured by default. Developers can use their existing container images, written in any language. Since Cellery is a code-first approach, it is very simple to code and write cell file, without having to learn several Kubernetes constructs and write hundreds of lines of YAML files. Therefore, it quickly helps to onboard developers into Kubernetes and have a controlled microservices architecture based on cells (Cell-Based Architecture).

What is K3s?

It is also known as 5 less than Kubernetes. Basically, it is a light weight Kubernetes with removed legacy, alpha, non-default features, and in-tree plugins (cloud providers and storage plugins) which can be replaced with out-of-tree add-ons. And hence it provides a binary of less than 40 MB and only 512 MB of RAM required to run. Therefore, it is very suitable for developers to install K3s within the local machine with limited resources and get the same functionality as of Kubernertes cluster.

Install Cellery on top of K3s

Let us get started on the steps for installing Cellery on K3s. As a first step, we need to install K3s, and K3s can be directly installed in Linux, but for macOS, we have to install VM and then can install K3s on top of it.

1.1. Install K3s in Linux

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-deploy traefik" sh -
# Check for Ready node, takes maybe 30 seconds
k3s kubectl get node

1.2. Install K3s in MacOS

As we cannot install the K3s directly onto macOS, we have to install a VM and then install K3s on top of it. For this, I am going to use multipaas in this article.

brew search multipass
brew cask install multipass

Once multipass is installed, let us create a VM for K3s with 4G memory, and 10G disk space as we need to install Cellery on top of the K3s.

multipass launch --name k3s --mem 4G --disk 10G

Wait until the k3s VM created, then open a shell to the VM as shown below.

multipass shell k3s

Execute below command in the shell of the k3s VM, to install the K3s into it. Note, this is the same command to install the K3s on Linux.

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-deploy traefik" sh - 
# Check for Ready node, takes maybe 30 seconds

In order to verify the k3s is working as expected, execute below commands.

sudo chown -R "$USER:" /etc/rancher/k3s/k3s.yaml
k3s kubectl get node

2. Configure host machine’s kubectl to k3s cluster

This steps are required if you have installed the k3s by using multipass in macOS. I assume, you have already installed kubectl in your host machine for below operations. Execute below commands to add the kube config to the host machine.

multipass copy-files k3s:/etc/rancher/k3s/k3s.yaml ${HOME}/.kube/k3s.yaml
K3S_NODEIP="https://$(multipass info k3s | grep "IPv4" | awk -F' ' '{print $2}'):6443"
sed -ie s,https://localhost:6443,${K3S_NODEIP},g ${HOME}/.kube/k3s.yaml

Try out kubectl commands pointing to k3s cluster as shown below.

kubectl --kubeconfig=${HOME}/.kube/k3s.yaml get nodes
NAME   STATUS   ROLES    AGE   VERSION
k3s    Ready    master   43m   v1.14.5-k3s.1

We need to change the default config as the k3s cluster config to install cellery into k3s. Inorder to do that, execute below commands, and hereafer we do not need to pass the kubeconfig along with each command.

export KUBECONFIG=${HOME}/.kube/k3s.yaml
kubectl get nodes
NAME   STATUS   ROLES    AGE   VERSION
k3s    Ready    master   59m   v1.14.5-k3s.1

3. Install Cellery on k3s

Download Cellery, and install it in your machine. Linux users can install as below, and macOS users can just double click on pkg and install.

sudo dpkg -i cellery-ubuntu-x64-0.3.0.deb

Execute below command to install Cellery into the already running k3s cluster. Make sure you have already configured the current kubernetes config to k3s cluster by executing kubectl get nodes. We have already performed it in the step-2.

cellery setup create existing
✔ Creating cellery runtime
✔ Setting up cellery namespace
✔ Applying istio crds
✔ Creating ingress-nginx
✔ Enabling istio injection
✔ Installing istio
✔ Creating controller
✔ Configuring mysql
✔ Creating ConfigMaps
✔ Adding idp
✔ Checking cluster status...
✔ Cluster status...OK
✔ Checking runtime status (Istio)...
✔ Runtime status (Istio)...OK
✔ Checking runtime status (Cellery)...
✔ Runtime status (Cellery)...OK

We need to update the correct IP as the node port IP for the ingress-nginx. Extract the IP of the k3s cluster via below command.

multipass list
Name                    State             IPv4             Image
k3s                    Running           192.168.64.3     Ubuntu 18.04 LTS

Edit the ~/.cellery/k8s-artefacts/system/service-nodeport.yaml and add the IP you have retrieved from above command.

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  externalIPs:
  - 172.17.17.100
  - 10.0.2.15
  - 192.168.56.10
  - 192.168.99.100
  - 192.168.64.3
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

Apply the above changes on to the kubernetes cluster as shown below.

kubectl apply -f service-nodeport.yaml

Get the list of ingress in the cellery-system namespace and make sure the IP included in the above step is included.

kubectl get ingress -n cellery-system
NAME             HOSTS                ADDRESS        PORTS     AGE
wso2is-ingress   idp.cellery-system   192.168.64.3   80, 443   14m

Hurray!! We have successfully installed Cellery on K3s.

We have installed Cellery basic setup into K3s cluster with above steps. If you want to install complete setup, you have to increase K3s VM memory size to 8G and disk space to 20G.

4. Deploy hello world cell

Let us deploy a simple hello world cell and check whether the it works. This cell is a hello world web application, you can find more information on the sample here.

cellery run wso2cellery/hello-world-cell:latest -n hello-world-cell -y

Get the ingress IP of the hello-world.com as shown below and include hello-world.com hostname and corresponding IP to /etc/hosts.

kubectl get ingress
NAME                                HOSTS             ADDRESS        PORTS   AGE
hello-world-cell--gateway-ingress   hello-world.com   192.168.64.3   80      2m29s
cat /etc/host
192.168.64.3 hello-world.com

Now, go to URL http://hello-world.com/ .

That’s all! You are ready to play with Cellery and try it’s features with your locally deployed K3S cluster.

Conclusion

It is very easy to create a light weight kubernetes cluster with K3s in both Linux and MacOS. Cellery will work with any kubeternetes cluster, and hence once we get the kubectl on host machine to point to the intended K3s cluster, you just need to execute a single command to install Cellery.

Note, I have used K3s version 0.8.0, and Cellery version 0.3.0 to tryout and the instructions provided above is based on those versions.

It is time to try more on Cellery, visit below links to get more ideas about it.

• Cellery website
• Getting started guide
• Cellery samples