A Simple Guide for Python Packaging

Jie Feng
Jie Feng
Feb 19, 2017 · 3 min read

Code reuse is a very common need. It saves you time for writing the same code multiple times, enables leveraging other smart people’s work to make new things happen. Even just for one project, it helps organize code in a modular way so you can maintain each part separately. When it comes to python, it means format your project so it can be easily packaged. This is a simple instruction on how to go from nothing to a package that you can proudly put it in your portfolio to be used by other people.

Package Structure

When you create a ‘.py’ file, that is a module. You can define classes, functions, variables in that module named as the filename. When you put one or more modules in a folder and add a ‘__init__.py’ file, you create a package named as the folder. So a common package folder structure could be like this:

Image for post
Image for post
Example package structure from official python document.

Build a Package

Now that you have put your code in a package structure, let’s take a look at how to build a package. To do that, the most important thing is the setup configure file ‘setup.py’. You use that to specify metadata. Here is an example of the file:

from setuptools import setup, find_packages

setup(name='funniest',
version='0.1',
description='The funniest joke in the world',
long_description='Really, the funniest around.',
classifiers=[
'Development Status :: 3 - Alpha',
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 2.7',
'Topic :: Text Processing :: Linguistic',
],
keywords='funniest joke comedy flying circus',
url='http://github.com/storborg/funniest',
author='Flying Circus',
author_email='flyingcircus@example.com',
license='MIT',
packages=find_packages(),
install_requires=[
'markdown',
],
include_package_data=True,
zip_safe=False)

More details about setup function can be found .

Specify Dependencies

To make the package ready to work after installation, you want to tell the setup process what are the other packages it relies on so they can be automatically installed. Simple put that into ‘install_requires=[…]’ within setup function. You can optionally specify a version:

install_requires=[‘A==1.0’, ‘B>=1,<2’]

These dependencies are public packages which can be installed via pip. If you are relying on ones not on pypi, use this:

dependency_links=[‘http://github.com/user/repo/tarball/master#egg=package-1.0']

Add Additional Files

Often your package may need to include other types of files. To do that:

  1. Set ‘include_package_data=True’
  2. Create a MANIFEST.in file in the folder of setup.py
  3. Add file path to MANIFEST as ‘include /path/to/test.json’, the path is relative. You can also use search pattern like ‘*.json’.

Build Package

Now you have everything in place, let’s build the package. Building a package means creating a source code distribution (e.g. tar.gz) or a binary file (e.g. wheel). It’s simple to do.

python setup.py sdist bdist_wheel

This will create both format in ‘./dist’ folder. The files will be named based on the package name and version.

Distribute a Package

Congratulations! You now have a shiny package. Now it’s the time to make it available to other people or your future self. We are gonna upload it to Pypi so you can do your favorite ‘pip install’.

First, you need to create an account on pypi .

Then, simply run:

twine upload dist/*

Here we assume the files under ‘dist/’ will all be uploaded to pypi. You can of course specify a particular file. Twine is a simple tool to help us interacting with pypi. When you upload for the first time, it will try to register the package on pypi.

Use a Package

At this point, you have published your own package on pypi. To use that, just run pip install my_package to get it to your machine. Then use ‘import my_package’ in your code.

If you are developing the package locally and want to use it for test. Add the path to the package folder to $PYTHONPATH and it’s good to go. If you don’t want to touch the environment variable, just do pip install -e . in the root package folder and it will be installed just like a published one. Plus, you don’t need to do anything every time you make changes to your code, it will keep up-to-date.

Here are two great sources to read if you want a bit more details on python packaging:

Little Big Engineering

Short tutorials on various kinds of engineering and product…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface.

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox.

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store