Setting up a private Python package repository on Amazon S3

If you write software in Python, you’ve definitely used pip — Python’s package manager — before. Pip — a recursive acronym for “pip installs packages” — allows you to install Python packages that are publicly available.

Pip defaults to searching in and downloading from PyPI, the official package repository managed by the Python Software Foundation. PyPI is great for publicly available open source packages, such as the one we’re open sourcing today, but it’s not the go-to choice for distributing private or closed source packages. For that purpose, you should use a privately hosted package repository. There are a number of tools available on this thing called the ‘internet’, such as Gemfury (commercial) or pypiserver (open source). Commercial or opensourced, all of these options require you to either run your own server or pay for someone else to do that for you.

At November Five, we use many of Amazon’s Web Services, such as EC2, RDS, ElastiCache, SQS, SNS, SES, Route 53, Lambda, DynamoDB, Cloudformation, Cloudfront, and of course, S3. S3 (Simple Storage Service) is an online service for file storage that allows you to store terabytes of data, but can also be used for static hosting. It has the double advantage of being both low in maintenance and cheap; you’d only be paying a couple of cents for the storage and bandwidth used by your site.

The tool we’re opensourcing today, s3pypi, allows you to more easily create your own package repository on S3.

Use it and never let private package repositories make you say NI again.

You can read the full tutorial in our blog post!

If you found this post helpful, click the 👏👏👏 button or share the article, so your friends can benefit from it too!