Where is this “Orchestration” thing going?
Orchestration is the act of creating and managing the resources, workloads, configuration, and data in an environment or set of environments in support of business operations and production. Orchestration is traditionally performed by system operators or administrators, but in the last decade automated orchestration systems have become robust and increasingly autonomous over the last 4 years.
Automated orchestration systems — such as Kubernetes — provide high-level abstractions and languages for describing operations on (or the desired state of) those resources under orchestration. Those same systems facilitate well-defined subprocesses or workflows that require repeatability and consistency. In doing so, they unload operators from mundane task work and reduce operational risk from modeled process violations (making mistakes while performing modeled processes).
Humans generally neither enjoy, nor are particularly well-suited for repetitive work that requires consistency.
Unmodeled error states (how things break) and unmodeled process violations (mistakes in processes that exist but are unmodeled) are still a risk. That risk has motivated further development and integration of resource telemetry with orchestration tooling in pursuit of more robust automated response capabilities. These systems will continue to advance in three key ways.
First, developers will increase breadth and resolution of application and resource telemetry. Doing so will provide greater insight into the current state of a system. Improving the quality and quantity of data gathered about an operational system will improve the resolution and completeness of system and process models. It reduces gaps and helps eliminate the unmodeled domain.
Second, orchestration will advance by improving specificity for system desired state, and expanding the reach of orchestration with new resource types. Expanding the orchestration vocabulary allows both human and autonomous operators to communicate state with improved precision and expanded semantic power. Concretely this looks like “more resources” in Kubernetes land, or at least more attributes that lend to modeling more states.
Last, developers will augment and refine the operational knowledge encoded in autonomous operators and develop higher-level abstractions to empower and simplify the experience for users. This enhancement will help automated reconciliation between desired and realized state achieve greater fidelity, but depends on the first two advances.
Automatons encoded with operational knowledge will unburden human operators and at lower cost than managed solutions.
Today our orchestration systems are generally application-agnostic. Meaning they don’t know much about the applications that they are responsible for. But that is changing quickly. And in some ways the advanced described above are already taking place.
Target applications like databases are already instrumented well enough to accommodate advanced orchestration tooling.
Many Kubernetes Operators have been released to model and orchestrate specific applications and their associated management life cycles. Consider the MySQL Operator provided by Oracle: https://github.com/oracle/mysql-operator or anything on this list of other popular operators: https://github.com/operator-framework/awesome-operators.
This field has come a long way in the previous decade since the dawn of the API era. More individuals than ever have access to broadband and more businesses than ever depend on 24x7 operations. These things are only going to get more intense and I’d expect work in this area to continue with the same vigor.
