Share the (PixieDust) Magic
Packaging and distributing your PixieDust plugin as a Python module on PyPI
Previously, you learned how to create your own custom visualization for PixieDust. The visualization code was written and tested directly in notebook cells, which is great for proof-of-concept and rapid iterating. However, it may not be the best vehicle for packaging and distributing your work to a large audience.
Magic is best when it can be experienced by everyone. To get your visualization out there, you will need to take it from notebook cells to a Python module, and then to a software repository for others to download and use.
To make sure your PixieDust magic is more easily consumable by others, you first need to package it up as a Python project.
The Python Packaging User Guide outlines instructions on packaging your Python project. However, you can use the
generate command of the PixieDust installer to simplify the process and automatically set up the scaffolding for your project.
While the instructions here use the Simple Word Cloud visualization discussed in the previous post, you can follow along with any PixieDust visualization you have created. These instructions also presume that you already have a local Jupyter environment, as well as a local install of PixieDust.
To get started, run the command
jupyter pixiedust generate from a terminal window and answer the subsequent prompts. Be sure to enter a name for your project that will not conflict with an existing Python package. For the
project type, choose option 1 (i.e., 1.Display Visualization).
Once completed, you will see a listing of the files created.
If you receive a TemplateNotFound error, uninstall PixieDust via
pip uninstall -y pixiedustand reinstall it using
pip install -e <path_to_pixiedust_dir>updating <path_to_pixiedust_dir>.
The result of the
generate command is a Python project with a default PixieDust visualization. You can now edit this default visualization, updating it with the code from the Simple Word Cloud.
The code for the template and metadata of the Simple Word Cloud in the previous post will be placed into separate files for this Python project. More specifically, you will need to update these files:
- helloWorld.html — Replace its content with the Simple Word Cloud’s template HTML fragment. Also, replace the
- display.py — Update the
doRenderfunction with that from the Simple Word Cloud and then replace the
The HTML fragment is moved to its own file; therefore,
doRendershould instead call
self._addHTMLTemplatewith the file name and pass in any desired parameters.
Do not forget to include the appropriate imports near the top of display.py:
from wordcloud import WordCloud
- __init.py__ — Update the
getMenuInfofunction with Simple Word Cloud’s implementation.
- setup.py — Update the
install_requiresto include the
wordcloudmodule and set
- MANIFEST.in — Add a line to have template files included in the distribution:
recursive-include pixiedust_wordcloud templates/*
With your Python project set up, you can install and run it from the safety of your local Jupyter environment. From a local notebook cell run:
!pip install -e <full_path_to_your_project>
Replace <full_path_to_your_project> accordingly and restart the kernel before continuing.
To test your new module, in a new cell run:
df = pixiedust.sampleData(7)
df2 = df.groupBy("street").count()
Be sure to import the appropriate name of your project.
Select Simple Word Cloud from the charts dropdown menu.
Congrats! You have turned your PixieDust visualization into a consumable Python module.
Now it’s time to put your magic on center stage for all to enjoy.
PyPI (Python Package Index) is the official third-party software repository for Python. It’s a go-to resource for finding Python modules.
Before you can proceed to including your project in PyPI, be sure to have:
- an account registered on PyPI
- twine installed (from a terminal window:
pip install twine)
The first step is to create a source distribution, which would be uploaded to PyPI. From a terminal shell, at the root directory of your project, run the command:
python setup.py sdist
If any errors occur in the output, resolve them, and rerun the command.
The next step is to take the distribution output and upload to PyPI. From the terminal shell (at the root directory of your project), run:
twine upload dist/*
Enter your PyPI credentials when prompted.
Congrats again! You have published your PixieDust visualization to the world. Learn more about customizing your packaging and distribution with various configuration options in Python’s documentation.
Your visualization should be available on PyPI for anyone to download, install, and use. Go ahead, return to your notebook and give it a try.
Begin by uninstalling the local version you had installed earlier. In a notebook cell, run:
!pip uninstall -ypixiedust_wordcloud
Then install your visualization directly from PyPI. In a notebook cell, run:
!pip install pixiedust_wordcloud
Restart the kernel and rerun the test cell from earlier.
Presto! You have yourself a PixieDust visualization that you, your friends, and total strangers can all easily download and use.
With no smoke or mirrors, you have created (PixieDust) magic! You have made data visualization easier, and it is worth sharing. Don’t hesitate to put it out there—inform the PixieDust community! Someone might be looking for exactly what you created.
If you enjoyed this article, please ♡ it to recommend it to other Medium readers.