Share the (PixieDust) Magic

Packaging and distributing your PixieDust plugin as a Python module on PyPI

https://unsplash.com/@yvettedewit?photo=NYrVisodQ2M

Previously, you learned how to create your own custom visualization for PixieDust. The visualization code was written and tested directly in notebook cells, which is great for proof-of-concept and rapid iterating. However, it may not be the best vehicle for packaging and distributing your work to a large audience.

Magic is best when it can be experienced by everyone. To get your visualization out there, you will need to take it from notebook cells to a Python module, and then to a software repository for others to download and use.

The props

To make sure your PixieDust magic is more easily consumable by others, you first need to package it up as a Python project.

The Python Packaging User Guide outlines instructions on packaging your Python project. However, you can use the generate command of the PixieDust installer to simplify the process and automatically set up the scaffolding for your project.

While the instructions here use the Simple Word Cloud visualization discussed in the previous post, you can follow along with any PixieDust visualization you have created. These instructions also presume that you already have a local Jupyter environment, as well as a local install of PixieDust.

To get started, run the command jupyter pixiedust generate from a terminal window and answer the subsequent prompts. Be sure to enter a name for your project that will not conflict with an existing Python package. For the project type, choose option 1 (i.e., 1.Display Visualization).

Once completed, you will see a listing of the files created.

Output from command: jupyter pixiedust generate

If you receive a TemplateNotFound error, uninstall PixieDust via pip uninstall -y pixiedust and reinstall it using pip install -e <path_to_pixiedust_dir> updating <path_to_pixiedust_dir>.

The setup

The result of the generate command is a Python project with a default PixieDust visualization. You can now edit this default visualization, updating it with the code from the Simple Word Cloud.

The code for the template and metadata of the Simple Word Cloud in the previous post will be placed into separate files for this Python project. More specifically, you will need to update these files:

• helloWorld.html — Replace its content with the Simple Word Cloud’s template HTML fragment. Also, replace the {0} with {{base64image}}:

<center><img src="data:image/png;base64,{{base64image}}"></center>

• display.py — Update the doRender function with that from the Simple Word Cloud and then replace the self.__addHTMLTemplateString call with:

self._addHTMLTemplate("helloWorld.html", base64image=img_str.decode("ascii"))

The HTML fragment is moved to its own file; therefore, doRender should instead call self._addHTMLTemplate with the file name and pass in any desired parameters.

Do not forget to include the appropriate imports near the top of display.py:

from wordcloud import WordCloud
import cStringIO
import base64

• __init.py__ — Update the getMenuInfo function with Simple Word Cloud’s implementation.
• setup.py — Update the install_requires to include the wordcloud module and set include_package_data to True:

install_requires=['pixiedust', 'wordcloud'],
include_package_data=True,

• MANIFEST.in — Add a line to have template files included in the distribution:

recursive-include pixiedust_wordcloud templates/*

The rehearsal

With your Python project set up, you can install and run it from the safety of your local Jupyter environment. From a local notebook cell run:

!pip install -e <full_path_to_your_project>

Replace <full_path_to_your_project> accordingly and restart the kernel before continuing.

To test your new module, in a new cell run:

import pixiedust
import pixiedust_wordcloud
df = pixiedust.sampleData(7)
df2 = df.groupBy("street").count()
display(df2)

Be sure to import the appropriate name of your project.

Select Simple Word Cloud from the charts dropdown menu.

Congrats! You have turned your PixieDust visualization into a consumable Python module.

PixieDust Word Cloud Visualization

Now it’s time to put your magic on center stage for all to enjoy.

The stage

PyPI (Python Package Index) is the official third-party software repository for Python. It’s a go-to resource for finding Python modules.

Before you can proceed to including your project in PyPI, be sure to have:

• an account registered on PyPI
• twine installed (from a terminal window: pip install twine)

The first step is to create a source distribution, which would be uploaded to PyPI. From a terminal shell, at the root directory of your project, run the command:

python setup.py sdist

If any errors occur in the output, resolve them, and rerun the command.

The next step is to take the distribution output and upload to PyPI. From the terminal shell (at the root directory of your project), run:

twine upload dist/*

Enter your PyPI credentials when prompted.

Congrats again! You have published your PixieDust visualization to the world. Learn more about customizing your packaging and distribution with various configuration options in Python’s documentation.

The show

Your visualization should be available on PyPI for anyone to download, install, and use. Go ahead, return to your notebook and give it a try.

Begin by uninstalling the local version you had installed earlier. In a notebook cell, run:

!pip uninstall -y pixiedust_wordcloud

Then install your visualization directly from PyPI. In a notebook cell, run:

!pip install pixiedust_wordcloud

Restart the kernel and rerun the test cell from earlier.

Presto! You have yourself a PixieDust visualization that you, your friends, and total strangers can all easily download and use.

The after-party

With no smoke or mirrors, you have created (PixieDust) magic! You have made data visualization easier, and it is worth sharing. Don’t hesitate to put it out there—inform the PixieDust community! Someone might be looking for exactly what you created.

If you want to look at a completed version of the pixiedust_wordcloud, you can clone the repo from GitHub or install the module from PyPI. Feel free to improve upon this version.

If you enjoyed this article, please ♡ it to recommend it to other Medium readers.