Creating and sharing private Python packages

An introduction to storing your private Python packages in Packagr

Christopher Davies

Published in

Packagr

6 min readOct 2, 2019

This article was updated on 2n October to reflect updates to the Packagr interface, functionality and plans

The problem

Python provides a convenient way of bundling your projects into packages, which can easily be distributed using PyPI and installed via pip. But what about your private packages? Anything you upload to PyPI is automatically made public and can be downloaded by anybody.

You may consider storing your built packages in your version control system, but it can be tricky to manage versions, and its annoying to have to repackage your code before every push. Alternatively, you could consider rolling your own private PyPI mirror, but this takes time, effort and cost.

This is where Packagr could provide a convenient alternative. Packagr is a secure, cloud hosted PyPI server where you can store your private Python packages, while retaining full control over who has access to what. Packagr also provides:

Convenient integration with CI/CD tools such as Gitlab CI and TravisCI
Security scans — Packagr checks alerts you to any known securiy issues in the dependencies used by your packages
Fine grained control of package access
Automatically control how many versions of your packages are available for download, so you don’t get a pip list that looks like this:

The competitors

Full disclaimer — There are a number of other similar solutions to Packagr out there. They range from monumentally expensive solutions such as packagecloud and cloudsmith (which start from $75/$99) a month to more basic services such as Gemfury, which doesn’t provide a security scanner.

Packagr aims to fill the gap here — it provides all the functionality you need at a competitive price

Using Packagr

This is a quick tutorial to get you started. Keep in mind this isn’t the only way to deploy packages to Packagr, its just the most basic. For more advanced topics, please refer to our other tutorials

Create a Python package

First of all, lets create a python package, if you don’t have one already. Use the following structure to create a simple hello world app

hello/
    helloworld/
        __init__.py
    setup.py

Paste the following code into __init__.py

def hello():
    return "Hello, World!"

And in your setup.py:

from setuptools import setupsetup(name='my-awesome-package',
      version='0.1',
      description='Test',
      author='Me',
      author_email='me@example.com',
      license='MIT',
      packages=['helloworld'],
      zip_safe=False)

To create the package, run the following command in a terminal:

python setup.py sdist

This will create a folder called dist containing a package called helloworld-0.1.tar.gz:

hello/
    dist/    
        helloworld-0.1.tar.gz
    helloworld/
        __init__.py
    setup.py

This is the package that we are going to push to our private server.

Sign up for Packagr

Next, you need to create an account at Packagr, by clicking on the Sign up button in the top right hand corner, and following the simple process. You’ll be emailed a code to verify your email address.

When you first sign up, you’ll be added to the free tier. This allows you to create public packages, which is fine for the purposes of testing Packagr out. Just keep in mind that you’ll need to give your package a unique name if you’re using the free tier — there are already packages in there called ‘hello world’

If you’re ready to take out a paid plan now, you can do this by clicking on Account settings when you log into your account. Paid tiers start from £10 per month and add private packages, more storage, support for additional users and security scans

When you first log in to Packagr, you will be taken to your package list (yours will be empty for now)

Click on the Create a new package button to see the details of your private cloud. This will show you the commands you need to use to upload your package

We now have somewhere to upload our package to. CD into the dist folder that we created earlier, and copy and paste the above commands into your terminal. You will be prompted for credentials — enter the email address and password you used to setup your Packagr account

$ cd dist
$ pip install twine
...
Successfully installed certifi-2018.4.16 chardet-3.0.4 idna-2.6 pkginfo-1.4.2 requests-2.18.4 requests-toolbelt-0.8.0 tqdm-4.23.4 twine-1.11.0 urllib3-1.22$ twine upload --repository-url https://api.packagr.app/63cdQSDO/ *
...
Uploading helloworld-0.1.tar.gz
100%|█████████████████████████| 3.34k/3.34k [00:00<00:00, 8.23kB/s]

If all works correctly, you should be able to refresh your page and see the package you just pushed:

Clicking on the package shows the installation command for it:

Running this command in a terminal installs your package. If you chose either of the paid tiers, then anything you upload will be marked as private by default, and you will be prompted to enter your password. If you’re using the free tier, all packages are uploaded as public, meaning they can be installed by anybody who has your index url

$ pip install helloworld -i https://api.packagr.app/63cdQSDO/
...
Successfully installed helloworld-0.1

Notes

If using the paid tiers, all packages uploaded to Packagr will be marked as private by default. You can either mark them as public to make them available to anybody, or share your package with other specific people using Access tokens. These are explained in the docs
If you only signed up for the free tier, you won’t actually be given a Hash ID — you’ll need to use the public endpoint (the above process doesn’t change)
If your package has additional dependencies from PyPI, you should include your repository URL using the extra-index-url argument, instead of the -i switch. Doing this will treat your private cloud repository as an additional index, instead of the only index
You can create a pip.conf file to save having to enter credentials for every command, as shown in the below example. (note that the username:password string is only required for private repositories)

[global]
extra-index-url = https://myusername:mypassword@api.packagr.app/my-cloud-id/

Next steps

If you’d like to do more with Packagr, then you might like to look at integrating it with an alternate approach for building packages, such as Poetry. Alternatively, you might want to see how you can integrate with CI/CD providers such as TravisCI or Gitlab CI

Is there another Packagr tutorial you’d like to see? Feel free to suggest it in the comments