The “Orchestration Pyramid” — It’s hard managing a mesh with multiple tools. Some people will argue three layers is too many. Some, too few.

What is Data Orchestration and why is it misunderstood?

Data Orchestration has often been confused with workflow orchestration. Workflow orchestration is the process of triggering and monitoring the status of tasks; nothing more, nothing less.

Preface

This post is part of our long-reads series. I’m often asked what Orchestra does and how it fits in the landscape we have today. I, alongside many actual thought leaders, believe there is a big difference between “data orchestration” — the process of reliably and efficiently moving data into production, and workflow orchestration. In here, we dive into what orchestration is, how these concepts interact with it, and take a closer look at why some platforms offer orchestration capabilities but are actually a bit more. This is all tied to the concept of an “Orchestration Pyramid”, which is just a diagrammatic representation of how a complete data architecture might look. This first appeared on the Orchestra Blog.

Introduction

Data Orchestration has often been confused with workflow orchestration. Workflow orchestration is the process of triggering and monitoring the status of tasks; nothing more, nothing less. Its essence is the batch or…