Scaling of ML & AI Use Cases
One core element in every ML & AI project as well as in the Machine Learning Incubator is the ability to efficiently scale ML & AI use cases. A suitable governance model enables an efficient implementation of different ML & AI use cases into production. This process starts with a use case funnel management and enables the collaboration from the idea to the product. Successful scaling — towards achieving a higher number of implemented use cases — requires the organization to deal with not only limiting technological factors, but also data governance issues. Moreover, legal approval processes need to be largely automated and internal processes adapted to repetitive development cycles. The collaboration model, already addressed in previous articles, needs to be expanded to include a comprehensive concept of rights and roles as well as a data catalogue to enable ML & AI projects to be implemented across different departments while adhering to data-protection compliance.
From a strategic perspective, new approval and quality gates are set up for control purposes in order to create a uniform ML & AI requirement and implementation structure. Furthermore, specific success indicators as acceptance criteria for new ML & AI projects are determined, while agile working methods are introduced to the organization. In this step, the ML Incubator focuses on a support of all initiatives through selective coaching, expansion of synergies between different teams and departments as well as establishment of a full cost transparency. In addition, it is important that the ROI and TCO of ML & AI initiatives are integrated into existing reporting structures, creating a transparent communication that encourages further support from management on future ML & AI initiatives.
Further recommendations to reduce complexity and improve traceability and clarity include the development of a central database-driven metadata management for ML & AI projects.
This system stores information on project requirements and the development and deployment status of projects, the data sources, data preparation pipelines and model types used, as well as the people involved in the development. It thereby forms an important basis for efficient collaboration within the company and the successful use of shared data pipelines — in addition to avoiding multiple implementations of the same logic. The metadata from that system can also be used to generate parts of a data pipeline or to automatically parameterize generic loading routes. With the help of suitable visualizations (e.g. Dataflow Overview, Data Lineage, Impact Analysis), a general overview is facilitated and the onboarding time for new project staff is shortened. In addition, concrete ML & AI projects are unified by a common management of metadata and codes so that synergies become usable.
The metadata management can technically be distributed across several interlinked systems, e.g. JIRA for the management of requirements (at epic and story level), a manually maintained relational database for the management of projects, data sources and data preparation steps, open source systems such as MLFlow for the management of models and training processes, and, for example, Seldon for automatic deployment to a Kubernetes-based productive environment. To improve operability, the monitoring of ETL processes, server resources, label quality, etc. will be standardized across projects, thus reducing project-specific development and maintenance efforts.
After implementing the scaling measures, a significant reduction in the development time for new ML & AI projects is ensured. Furthermore, to secure a full cost transparency across departments, our Incubator workshops and our projects include the ROI and TCO calculation of the ML & AI initiatives. Including these calculations in the overall business case and integrating them into existing reporting structures through the program encourages management support for further ML & AI-initiatives. The Machine Learning Incubator supports our clients through selective coaching to expand synergies between departments and teams.
