What is PyPI?
The official docs:
The Python Package Index (PyPI) is a repository of software for the Python programming language.
PyPI is the official 3rd party software repository for the Python language.
To install packages from the PyPI repository, you will need a package installer. The most common and recommended one is pip.
PyPI is public, anyone can access it and download any package listed there. However, sometimes you want to create your own private repository, mainly for security reasons.
To set up a PyPI server, you will need a server — it can be your own machine, your friend’s, or a machine in the cloud (AWS, GCP, …). In this tutorial, I assume that you already have a server that will host your private repository.
Since we’ll use Docker to run the server, we need to install Docker:
PyPI Server Setup
Let’s create a new directory for our PyPI configurations.
# replace with your name
Note that we are currently allowing unauthorized access by specifying the
-P . -a . command.
Now, we should configure our NGINX server. From within
pypi-server, create the following directories:
mkdir nginx nginx_cache
conf.d with the following content:l
We are listening on port 80 (default port), and passing all requests to “pypi-server:8080”, where our PyPI server is running.
By default, Compose sets up a single network for our application. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name. That’s why we can use the
pypi-server host in the configuration above.
Finally, let’s add (the default)
nginx.conf in the
nginx directory, which includes our custom configuration above (line 27):
Starting the Server
By now, we should be able to run the following from the
docker-compose up -d
Accessing the home route from the server:
$ curl localhost:8080
<title>Welcome to pypiserver!</title>
Welcome to pypiserver!
This is a PyPI compatible package index serving 11 packages.
To use this server with <code>pip</code>, run the following command:
<code>pip install --index-url http://pypi-server:8080/simple/ PACKAGE [PACKAGE2...]</code>
To use this server with <code>easy_install</code>, run the following command:
<code>easy_install --index-url http://pypi-server:8080/simple/ PACKAGE [PACKAGE2...]</code>
The complete list of all packages can be found <a href="/packages/">here</a> or via the <a href="/simple/">simple</a> index.
This instance is running version 1.4.2 of the <a href="https://pypi.org/project/pypiserver/">pypiserver</a> software.
pip, you can now download packages from your own private repository!
pip install --index-url http://<IP>:8080 my_package --trusted-host <IP>
Publishing packages to our newly created server can be done using Twine. Go ahead and check its documentation for further information.
While we are now working with our private repository, anyone who has access to our server can interact with the repository. To protect it, we can use
htpasswd. Install the
sudo yum install -y httpd-tools
Now, create a folder for authentication:
and create a username:
# create .htpasswd file and force SHA encryption
htpasswd -sc .htpasswd pypi-user
you will be prompted to insert a password. Once done,
.htpasswd will create an entry for the new user, with a hashed password:
We should next update our Docker compose file:
Note that we changed the command to require authentication for the update, download, and list commands. You can check the manual of the
pypi-server understands the following options:
-p, --port PORT
Listen on port PORT (default: 8080).
-i, --interface INTERFACE
Listen on interface INTERFACE (default: 0.0.0.0, any interface).
-a, --authenticate (update|download|list), ...
Comma-separated list of (case-insensitive) actions to authenticate.
Requires to have set the password (-P option).
To password-protect package downloads (in addition to uploads) while
leaving listings public, use:
-P foo/htpasswd.txt -a update,download
To allow unauthorized access, use:
-P . -a .
Note that when uploads are not protected, the `register` command
is not necessary, but `~/.pypirc` still need username and password fields,
even if bogus.
By default, only 'update' is password-protected.
Accessing the PyPI server will now require authentication.
Sometimes you don’t want to host your packages in a public repository. Creating a custom PyPI server is a good idea for increasing security and not exposing your packages to everyone.
Using NGINX, we introduced a caching mechanism, and we can easily configure a load-balancer when needed.