Obsidian and Jupyter Notebooks

Personal Knowledge Management for Data Science

Motivation

Jupyter notebooks, or more properly iPython notebooks, are fantastic tools for data exploration and modeling. You can run bits of code, interspersed with blocks of markdown, allowing you to easily work with data and present analyses and forecasts in a visual and interactive format. Notebooks can be easily shared via GitHub, or run on-line with Colab.

The problem with iPython notebooks, from a Personal Knowledge Management perspective, is they end up being “books on the shelf”. For Obsidian users and PKM practitioners, the whole point is to get away from keeping information in notebooks, with all the uselessness that that implies. One could, as you will see, just do data exploration directly in Obsidian instead of in a standard iPython notebook. But Obsidian is not an IDE. The ecosystem around the interactive Python notebook IDEs, be it Jupyter Lab itself, or VS Code, etc, is so useful, that working in Obsidian while exploring data and creating models would be unacceptably tedious. Using fit-for-purpose tools is very important for efficiency.

We need a painless way to capture all this information in our vault, making notes out of the notebooks. How can we easily make this information future-useful, without repeating/duplicating our efforts or doing a ton of copy/paste? In this article, I’ll explain the solution that works well for me, ensuring Obsidian-speed access to any bits of information I have in my notebooks, as…