Evolution of experiment tracking in Kedro
Kedro is an open-source Python toolbox that applies software engineering principles to data science code, making it easier to transition from prototype to production. Created by QuantumBlack, AI by McKinsey, it was open-sourced in 2019, and donated to the Linux Foundation in 2021.
Kedro is constantly being improved and in recent sprints we have expanded its experiment tracking capabilities in recognition of user feedback and product metrics that showed its popularity.
We had already integrated experiment tracking into Kedro-Viz, an interactive development tool which enables you to visualize the data, nodes and data pipelines of your data. In November 2021, we shipped the first iteration of experiment tracking in Kedro-Viz, which enabled users to see and compare different metrics, or tracked datasets, from their Kedro runs.
However, since the initial launch of experiment tracking, we identified further priorities for our users:
· Linking plots to an experiment — allowing users to save plots alongside their experiment metrics for easy comparison.
· Plotting experiment metrics derived from pipeline runs — enabling users to evaluate metric trade-offs and identify the best performing experiment.
· Writing experiments to a remote server — users share their experiments with other team members, encouraging multi-user collaboration within a team.
What’s new in Kedro experiment tracking?
We have recently released two exciting new features designed to address the first two priorities identified in our user research.
The first enhancement enables users to track their plots as part of their experiments and visualize them on Kedro-Viz experiment tracking. Users can then compare these plots side-by-side between runs, selecting up to three runs for easy comparison.
The second enables users to plot experiment metrics from pipeline runs in a time-series or parallel coordinate plot. Users can select, plot and compare multiple metrics and select the best performing experiment.
The next step is further evolution of Kedro to enable users to write experiments to a remote server, as opposed to storing their runs on a local machine. Existing Kedro users have indicated that this next progression would transform their workflow on Kedro.
You can find out more about experiment tracking in Kedro from our documentation, and check out our experiment tracking demo to explore the capabilities of Kedro-Viz further.
With the help of the QuantumBlack Labs team and the open-source community, Kedro and Kedro-Viz continue to evolve. Why not join our Slack community or check our website to stay up to date with the latest developments?
Authored by: Nero Okwa, Product Manager for Kedro-Viz
