MayaOnline — A Synergized Tool to manage storage for stateful workloads on Kubernetes
August 27, 2018
In October 2017, the term MayaOnline was coined for the first time at MayaData Inc. With the growing usage of our open-sourced OpenEBS Container Attached Storage (CAS) engine for stateful workloads in Kubernetes, there was a need to visualize, monitor and manage the storage components of the various Kubernetes clusters, where our customers were deploying OpenEBS engine. We began our initial experiments on building a prototype for an enterprise SaaS platform and launched an alpha version in December 2017 at the Kubecon event.
The initial version of the product had the following features:
- Import an existing OpenEBS-enabled K8s cluster.
- A dedicated Prometheus engine which collected the storage capacity and I/O metrics from the OpenEBS volumes in a time-series format.
- A dedicated alert-manager which configured the rules for the generation of alerts whenever the metrics reached the threshold limits.
- A dedicated Grafana service with industry-standard dashboards for visualizing the metrics.
- A dedicated chat-ops server which was integrated with Slack so that the cluster configuration and real-time alerts could be sent to the administrator’s slack channels.
We opened the portal for reviews and our users provided encouraging feedback which led us to launch the beta version by mid of May 2018 with additional features:
- Integration with Weave Scope for better visualization and easy sifting through the various K8s resources including storage support in Weave Scope.
- Support for OpenEBS-enabled OpenShift clusters.
- A multi-tenant Prometheus and Grafana engine.
- Additional dashboards for aggregate metrics.
- A dedicated chatbot called MuleBot.
- Enhanced user interface.
In the last couple of months, OpenEBS has evolved tremendously. A new open-source initiative i.e. Node Disk Manager (NDM) has been started for common management of K8S disks (physical and virtual) for on-premise and cloud deployments. OpenEBS 0.7 (read blog) now supports cStor — a faster and more resilient storage engine for enterprise workloads, in addition to its existing engine Jiva. Then there is Litmus which introduces chaos to test the resiliency of stateful workloads in K8s (read Chaos Engineering).
Our users are really excited with these developments and we are providing them with a new version of MayaOnline which integrates all of the above for providing data agility (read MDAP). Some of the cool new features include:
- End-to-end storage visualization.
- Ability to take snapshots and restore data from OpenEBS volumes.
- Launch Litmus chaos tests on K8s applications.
- Enhanced user interface.
Let me talk more about each of them.
End-to-End Storage Visualization
MayaOnline beta already had Scope integration where the user could get a holistic view of all the K8s storage constructs and resources like Persistent Volumes, Persistent Volume Claims and Storage Classes which she has configured to satisfy the storage needs of her workloads. The new MayaOnline goes one step further and integrates NDM into scope. Now a user can view/manage the connectivities between the PVs and the core storage components like StoragePools and Disks !! This provides the right perspective to the user to understand the complex relationships between the various resources which in turn facilitates easy management and debugging.
No other product in the market provides this level of control to the users.
Snapshots and Restore
Ability to take application-consistent snapshots has always been the need of the hour for the developers and they expect all storage solutions to make provisions for the same. Along with the E2E visualization, a MayaOnline user can now browse/navigate to the volume of his choice and take snapshots with a single click of a button. What’s more, the snapshots themselves can be viewed as K8s resources and support control operations like creating clones and restoring them !!
Litmus Integration
Chaos engineering is the buzzword these days and we at MayaData joined the bandwagon by developing Litmus — a-first-of-its-kind easy-to-develop Behavioral-Driven-Development Test framework where chaos tests can be written in plain English language !! Application developers and testers alike can use Litmus to induce real-time chaos like node …
