Data Version Control beta release: iterative machine learning

UPDATE: This blog post uses outdated DVC release 0.8. New tutorial — Data Version Control Tutorial. New project web site: dvc.org.

It is hardly possible in real life to develop a good machine learning model in a single pass. ML modeling is an iterative process and it is extremely important to keep track of your steps, dependencies between the steps, dependencies between your code and data files and all code running arguments. This becomes even more important and complicated in a team environment where data scientists’ collaboration takes a serious amount of the team’s effort.

Today, we are pleased to announce the beta version release of new open source tool — data version control or DVC. DVC is designed to help data scientists keep track of their ML processes and file dependencies in the simple form of git-like commands: “dvc run python train_model.py data/train_matrix.p data/model.p”. Your existing ML processes can be easily transformed into reproducible DVC pipelines regardless of which programming language or tool was used.

This blog post walks you through an iterative process of building a machine learning model with DVC using stackoverflow posts dataset. First, you should initialize a Git repository and download a modeling source code that we will be using to show DVC in action.

NOTE. THIS IS AN OUTDATED CODE FOR OLD DVC VERSION - 0.8. FOR NEWER DVC PLEASE USE CODE FROM THE NEW TUTORIAL.

The full pipeline could be built by running the bash code below. If you use Python version 3, please replace python to python3 and pip to pip3.

The one thing to wrap your head around is that DVC automatically derives the dependencies between the steps and builds the dependency graph (DAG) transparently to the user. This graph is used for reproducing parts of your pipeline which were affected by recent changes. In the next code sample we are changing feature extraction step of the pipeline and reproduce the final result. DVC derives that only three steps out of seven need to be rebuilt and runs these steps:

If you replace the input file that affects all the steps, then the entire pipeline will be reproduced.

Not only can DVC streamline your work into a single, reproducible environment, it also makes it easy to share this environment by Git including the dependencies (DAG) — an exciting collaboration feature which gives the ability to reproduce the research results in different computers. Moreover, you can share your data files through cloud storage services like AWS S3 or GCP Storage since DVC does not push data files to Git repositories.

The code below shows how to share your code and DAG through the Git and data files through S3:

Now, another data scientist can use this repository and reproduce the data files the same way you just did. If she doesn’t want (or has not enough computational resources) to reproduce everything, she can sync and lock shared data files. After that, only the last steps of the ML process will be reproduced.

The steps that were reproduced (red):

Conclusion

Thus, the model can be improved iteratively and DVC simplifies the iterative ML process and aids collaboration between data scientists.

We are very interested in your opinion and feedback. Please post your comments here or contact us in twitter — FullStackML.

If you found this tool useful, please “star” the DVC Github repository.