Conduit : Introduction
Our Production Engineering team knew it was time.
For close to thirty years, Blue Sky produced 12 feature films, a number of shorts and TV specials, and feature film visual effects on roughly the same pipeline with a few minor improvements over the years. An impressive feat, but the increased complexity and ambitions of the directors exposed the brittleness of an old code base.
Several studios including Rhythm & Hues and MPC have demonstrated successful implementations of modern pipeline concepts at SIGGRAPH. Fundamentals such as defining a “Task/Product” data relationship and a modern microservices backend provide remarkable improvements to user experience and data integrity. In 2015, with those concepts as a guide, the Blue Sky Production Engineering team went to the drawing board to build “Conduit”.
In this blog, we want to share with you what we’ve learned, built and developed. Where practical, we will share code for you to take and use as a guide or in your productions. We don’t claim that this is the definitive pipeline, just our crack at it.
Before diving into the stack we built, how much is just a pipe-dream and what’s actually production/battle-tested?
We deployed Conduit for two production use cases: tracking media and tracking asset health (aka QC). For the past year, Conduit has tracked image and media content along with metadata describing asset publishes on our active productions, Spies in Disguise (2019) and Nimona (2020). At SIGGRAPH 2018, Christopher Moore and Rebecca Hallac will discuss the Animation Performance and Asset Health initiatives that were built using Conduit as a backend (checkout their talk: Achieving and Maintaining Real-Time Rigs).
These use cases helped refine the scalability of our stack and hardware requirements. We are actively developing and deploying our new version control system, Product Version Control (PVC), for use on Nimona.
But before all that, we had to figure out how to identify stuff.
