Manage your data in an S3-compatible object storage with Scality CloudServer and Kubernetes

Zenko CloudServer (formerly S3 Server) is an Open Source project provided under the Apache License.
It provides a single unifying interface using the Amazon S3 API, supporting multi-cloud backend data storage both on-premises and in the Amazon S3 public cloud as well as other services, including: Available now for Amazon S3, Scality RING and Docker. Coming soon for other cloud platforms.
Deployable as a Docker container, Scality’s S3 Server transforms any storage space into S3-compatible storage. The software uses the same authentication model as the Amazon AWS S3 service. This allows a developer to develop an application based on local storage as if it were interacting with Amazon AWS S3.

First, you need to install the latest version of Docker.
An example with Linux Ubuntu (that I use in a VM VirtualBox)
Install Docker CE on Linux “Ubuntu”
https://docs.docker.com/install/linux/docker-ce/ubuntu/
sudo apt-get update && apt-get install docker-ce docker-ce-cli containerd.ioInstall minikube for a cluster with only one node (master and node)

Installation for minikube with option “none driver” on Linux VM
https://github.com/robertluwang/docker-hands-on-guide/blob/master/minikube-none-installation.md
Verify the installation
minikube status
minikube: Running cluster: Running kubectl: Correctly Configured: pointing to minikube-vm at 10.0.2.15
kubectl cluster-info
Kubernetes master is running at https://10.0.2.15:8443 KubeDNS is running at https://10.0.2.15:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use ‘kubectl cluster-info dump’.
kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
- kube-system coredns-c4cffd6dc-slx45 1/1 Running 2 5d
- kube-system etcd-minikube 1/1 Running 7 8d
- kube-system heapster-p2f9x 1/1 Running 6 5d
- kube-system influxdb-grafana-kspgm 2/2 Running 12 5d
- kube-system kube-addon-manager-minikube 1/1 Running 7 8d
- kube-system kube-apiserver-minikube 1/1 Running 7 8d
- kube-system kube-controller-manager-minikube 1/1 Running 7 8d
- kube-system kube-dns-79f5cdddc5–79sht 3/3 Running 41 2d
- kube-system kube-proxy-smk6n 1/1 Running 7 8d
- kube-system kube-scheduler-minikube 1/1 Running 8 8d
- kube-system kubernetes-dashboard-6f4cfc5d87-rs6dk 1/1 Running 13 8d
- kube-system storage-provisioner 1/1 Running 14 8d
Verify that the ADDON dashboard is enabled and set the ADDON heapster at enabled, then start the k8s console
minikube addons enable heapster
minikube addons list
- addon-manager: enabled
- coredns: enabled
- dashboard: enabled
- default-storageclass: enabled
- efk: disabled
- freshpod: disabled
- heapster: enabled
- ingress: disabled
- kube-dns: enabled
- metrics-server: disabled
- nvidia-driver-installer: disabled
- nvidia-gpu-device-plugin: disabled
- registry: disabled
- registry-creds: disabled
- storage-provisioner: enabled
Start the k8s console
minikube dashboard

Deploy CloudServer with Kubernetes
Refer to my GitHub repository for S3Server deployment files
https://github.com/Tigrou59/Scality/tree/master/S3-Server/k8s
In Dev environnment with k8s in cluster minikube
In production, we expect a cluster n nodes (3 nodes minimum) installed with kubeadm or provided by a cloud provider we also expect that data will be persistent. That you will use the multiple backends capabilities of Zenko CloudServer, and that you will have a custom endpoint for your local storage, and custom credentials for your local storage:
Many object k8s will be created, in logical order : ConfigMap that stores ENV variables
kubectl apply -f configmap-s3server.yml
kubectl get cm -o wide
NAME DATA AGE
- config-map-s3 3 1d
Persistent Volumes for storage local or on any other types of storage suported by kubernetes (Volume Plugin)
kubectl apply -f pv-s3server.yml
kubectl get pv -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
- s3-pv 1000Mi RWO,ROX Recycle Bound default/s3server-claim 3d
PersistentVolumeClaim for binding the PersistentVolume and then use refered this in pods
kubectl apply -f pvc-s3server.yml
kubectl get pvc -o wide
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
- s3server-claim Bound s3-pv 1000Mi RWO,ROX 3d
At the hight level, the Deployment wich including replicas, pods, labels, environment config, strategy of rolling update, claim for pvc…
kubectl apply -f deploy-s3server-pv.yml --record
kubectl get deploy -o wide
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
- s3server 3 3 3 3 1d s3server scality/s3server:latest app=s3server
kubectl get rs -l app=s3server -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
- s3server-6ffc956c55 3 3 3 30d s3server scality/s3server:latest app=s3server,pod-template-hash=2997512711
kubectl get pods -l app=s3server -o wide
NAME READY STATUS RESTARTS AGE IP NODE
- s3server-6ffc956c55-hp5cc 1/1 Running 1 1d 172.17.0.11 minikube
- s3server-6ffc956c55-llld7 1/1 Running 1 1d 172.17.0.10 minikube
- s3server-6ffc956c55-xhxbm 1/1 Running 1 1d 172.17.0.9 minikube
And finally the Service “NodeType” bring a stable and reliable network, DNS, load-balancing and expose your app outside of the kubernetes cluster, Cluster IP and Port (in our case : 10.109.72.126:30042 over TCP)
DNS-Based Service Recovery requires the DNS cluster — add-on, The DNS add-on implements a POD-based DNS service in the cluster and configures all kubelets (nodes) to use it for DNS ie: minikube addons list
kubectl apply -f svc-nodeport.yml
kubectl get svc -l app=s3server -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
- s3server-svc NodePort 10.109.72.126 8000:30042/TCP 5d app=s3server
In summary
By providing a free-standing implementation of the S3 API, CloudServer offers developers the freedom to build S3 apps and run them either on-premises, in the AWS public cloud, or both — with no code changes. CloudServer is deployed in a Docker container.
Behind the scene
Public Docker repository for cloudserver Scality Zenko
Public GitHub source repo
Zenko Cloud Server
Install minikube on a VM
Kubernetes documentation
Follow us on Twitter 🐦 and Facebook 👥 and join our Facebook Group 💬.
To join our community Slack 🗣️ and read our weekly Faun topics 🗞️, click here⬇

