Port your Jupyter Notebook Dependencies

Pip and Estella from Great Expectations

This will be a quick article, say I have this notebook:

from tqdm.notebook import tqd
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from dotenv import load_dotenv
import os
import seaborn as sns

load_dotenv()
api_key = os.getenv("API_KEY")
tqdm.pandas()

I cannot just send it to my buddy, and have Great Expectations that they can run it.

We need to provide the virtual environment and the dependencies therein.

PipReqs it!

pip install pipreqs

Pipreqs is a tool that analysis your python files (.py) in the given directory and compiles a requirements.txt with the exact versions.

But we have a notebook in the format of .ipynb! Enter nbconvert

pip install nbconvert

With this tool, we can now get a requirements.txt:

jupyter nbconvert --to script ./notebook.ipynb & pipreqs

With this run you’ll get a notebook.py (which you can remove), and the requirement.txt:

matplotlib==3.7.1
numpy==1.25.0
pandas==1.5.3
python-dotenv==1.0.0
seaborn==0.12.2
tqdm==4.65.0

Send it over with your notebook, and your team can create an environment using venv or conda with your python version and dependencies above.

Estella would be proud of this utility.

Conclusion

In this article, we used a combination of nbconvert and pipreqs to make our notebook portable.

References

• https://pypi.org/project/pipreqs/
• https://nbconvert.readthedocs.io/en/latest/index.html

Made with ❤ by Adam