Reinventing The Wheel: Publishing Your Own Python PIP

7 min readJan 31, 2024

Whether you have a real need to do this or you just want to show off things that most people already know (like I am doing here), you find yourself in the crossroads of wanting to publish your own python package. You know, like the ones that you type in your shell pip install my-big-python and then a whole lot of things happen with some fancy colors and progress bars and then you go to your code file and write from mybigpython import rubbersnake … yup, that (holy cow, that was a lot of “and” in one sentence. I vaguely remember my high school English teachers not being cool with it)

Although jokes aside there are real reasons to want to do that

You have a product, and these are the SDKs/Driver
You built something cool, and you can totally see people wanting a similar utility.
You want a distributed code base; you don’t want a mono-repo and you want to share the same functions without doing git clone all the time.
Or you want to be, as my co-pilot-in-cluster-f***s Danny says, “breh ! you now extra-official !!”

Whatever your reason is, if you have not done this, it can feel scary. But I gotchu. First thing you need to know is PyPI: Python Package Index. All the pip you have been using comes from here.

Go to https://pypi.org and create an account for yourself. This will be the account you will be using for all the pips you publish (Don’t forget to give yourself a nice little handle and lovely password like password01!).
After that, do the brotha’ a favor. Go to account settings and please enable 2FA.
While you are at it create API tokens that you will use to publish your package. You will need the API token once you enable 2FA anyway.

Now that’s out of the way, the important part -

Step 1: Go to ChatGPT
Step 2: Type “how to publish my own pypi”
Step 3: Follow the steps — and DONE!

ChatGPT gave me this. And honestly that’s basically it -

Create Your Python Package: Start by creating your Python package if you haven’t already... (blah blah). Ensure your package has a setup.py file, which is essential for packaging.
Prepare the Package Metadata: In the setup.py file, define the package metadata such as the package name, version, author, description, and other relevant information. This information is used to identify and display your package on PyPI… (blah, blah, blah!).

Wait no, this is actually important -

from setuptools import setup

setup(
    name="my-big-python", # this is the name in pip install <package-name>
    version="0.1", # this is VERY IMPORTANT. Everytime you make an update you need to up the counter
    author="The name yo momma gave you",
    description="This is a big python package",
    # This is the bare minimum but add more metadata as needed.

    # I have the following
    long_description="Some more text or copy-paste from your README.md.",
    long_description_content_type='text/markdown',
    keywords="keyword1 keyword2 keyword3", # 'openai tiktoken chatgpt chatbot',
    url='<github url or your site or your twitter handle. whatevs>', # i usually use the github url of the project repository
    author_email="The email you gave yourself",
    license="MIT", # or some other license
    packages=find_packages(),
    install_requires=[ # these are the pips that need to exist for your pip to function
        'numpy',
        'pandas',
        'icecream',
        'transformers',
        'scipy'
    ], 
    zip_safe=False    
)

Create a README and Documentation: Write a README.md file to provide documentation and instructions for users on how to use your package… (more blah blah)
Package Your Code: Make sure your Python package is organized correctly, and all the necessary files are included. You should have a directory structure that …

Okay this is also kinda’ important. You want your file structure to be something like the following -

my-big-python/
├── mybigpython/ # this is name in: import mybigpython
│   ├── __init__.py # indicates that the directory is a package
│   ├── rubber_snake.py # this is name in: from mybigpython import rubber_snake
│   └── gummy_worm.py
├── LICENSE
├── README.md
├── setup.py
└── optional_example.py # optional and can be in its own folder

Build a Distribution: Use the setuptools library to build a distribution package. Run the following command in your package’s root directory …

You can do this manually from your shell, which mean you will upload the content every time you have made changes. Run the following commands in your shell -

# Build the distribution package
python setup.py sdist bdist_wheel
# Install twine if you don't have it already. This is a tool to upload your package
pip install twine
# Now run twine to upload
twine upload dist/*
# You will be prompted to enter your PyPI credentials (username and password) during the upload process.
# Hoping you have enabled 2FA, you will need your API token
# as username: type "__token__" (NOT the lovely handle you created for yourself
# as password: copy-paste the API token

Or you can do this through GitHub actions where the code will automatically become a part of PyPI once the code is checked in. In both cases don’t forget to update the version number if your setup.py.

Verify Your Package on PyPI: Once your package is uploaded, you can find it on PyPI under your package name. Make sure to check if everything looks correct.
Share Your Package: Promote your package by sharing it with the Python community. You can share it on social media, forums, and platforms like GitHub to get more users and contributors.

Basically, make a post on LinkedIn how you solved world’s problems with this, write a blog on Medium (copy-paste your README.md), then post these links in twitter & reddit and then text all your friends to like all of your posts.

Maintain Your Package: Regularly update and maintain your package by fixing issues, adding features, and responding to user feedback.
That’s the basic step-by-step approach to publishing your Python package on PyPI. Remember to follow Python packaging best practices and guidelines for a successful and well-received package.

This cracked me up. You mean, I have to bug fix my own package and listen to people’s feedback? the AUDACITY! Those are not bugs. Those are FEATURES!

But before you go here some nuances to keep in mind -

License

Although not mandatory, you should have a file containing the license info. If you have not done this before, it is a text file with some words that stays in your repo. This talks about terms and conditions of how your code/library can be used and distributed. As for the wording, I am not a lawyer and neither did I consult with one. I generally use MIT license for my codes because I asked ChatGPT to generate a sample, and this is what I got. Also, because this one seems to have the least number of words that I think I understood. You can choose a bunch of other ones like GPL, Apache, BSD, MPL, LGPL etc. You can ask ChatGPT to write one for you and be done with it. Here is the sample.

Publishing/Distributing Through GitHub

You can publish using GitHub Actions. This way everytime you sync your code it will automatically trigger a workflow to do the publishing for you. You can even choose the branch the publish should happen through. This is my personal favorite approach (instead of twine-ing manually) but it needs some configuration. You can use one of the free workflows from the Github marketplace.

Go to your repo Actions tab and click New Workflow button and select the following. Keep in mind that this workflow requires you to have the long_description field specified in setup.py or else it will fail

Pre-built GitHub Action for Publishing in PyPI

Or you write your own action if you are masochistic enough. That’s what I did, and after that even water tasted like a fine single malt. Here is a sample in case if you are gung-ho. In both case you will need to upload your API token through repo Settings/Secrets and Variables

setup.py

version field is SUPER important for updates. If you don’t change the version number after every update, the publish WILL FAIL

description field is a quick snapshot

long_description is primarily a README. If you have the field, you will need to specify the long_description_content_type field. This is how it shows up

If you have nothing specified this is how it will look

README.md

Be nice and write all the beautiful thangs this package does. Write how you expect this to be used like SDK documentation. Or you can write “Ain’t nobody got time for this. Figure it out yourself” thats fine too. You can also refer the README.md content through your long_description field.

Tests and Examples

PAHAHAHA! TEST?! PAHAHAHA … we test in production baby #yolo

Yes, it is not mandatory but do include some examples or tests so that people can at least decipher how to use it.

Et, Vois la! You have your own python package. You are now extra-official!

PS

Previous article: Reinventing The Wheel: Deploying Go Code as Azure Web App or Function App | Medium

Same $hit somewhere else: Reinventing The Wheel: Publishing Your Own Python PIP | LinkedIn