Python 2.7 — pip issue installing almost any library

I’ve been working with Python 3 for a while, but recently I was involve in an automation project that required python 2.7.5. So, I installed Python2 and setup my virtualenv whith python 2 ready to start coding as a champ and when I was trying to import some libraries and I received some (InsecurePlatformWarning, SSLError, tlsv1 alert protocol version) errors 🤨.

After do some research about “Why I was NOT able to install Python packages” and received a [SSL: TLSV1_ALERT_PROTOCOL_VERSION]. I found the issue has two reasons:

1. Python 2.7 reach the end of its life on January 1st, 2020
2. TLS deprecation for pip on Apr 11th, 2018.

Upgrading pip using pip i.e pip install --upgrade pip will also not upgrade it correctly. It's just a chicken-and-egg issue. pip won't work unless using TLS >= 1.2.)

Python.org sites have stopped support for TLS versions 1.0 and 1.1. Here you have the Python status page:

Completed — The rolling brownouts are finished, and TLSv1.0 and TLSv1.1 have been disabled. Apr 11, 15:37 UTC

Update — The rolling brownouts have been upgraded to a blackout, TLSv1.0 and TLSv1.1 will be rejected with a HTTP 403 at all times. Apr 8, 15:49 UTC

The first step you can try to solve the issue is to: Run upgrade

curl https://bootstrap.pypa.io/get-pip.py | python

But if the curl command itself fails with error, or "tlsv1 alert protocol version" persists even after upgrading pip, it means your operating system's underlying OpenSSL library version<1.0.1 or Python version<2.7.9 (or <3.4 in Python 3) do not support the newer TLS 1.2 protocol that pip needs to connect to PyPI. You can easily check it in Python interpreter with this commands:

>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 0.9.8o 01 Jun 2010'
>>> ssl.PROTOCOL_TLSv1_2
 AttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'

The AttributeError (instead of expected '5') means your Python stdlib ssl module, compiled against old openssl lib, is lacking support for the TLSv1.2 protocol (even if the openssl library can or could be updated later).

Fortunately, it can be solved without upgrading Python (and the whole system), by manually installing extra Python packages.

Detailed step-by-step guide 😎

Step 1 — Download

Download the following packages from Python Packing Index (pypi.org) via your web browser of choice — choose recent wheels (.whl) for your OS/platform:

pip, asn1crypto, enum34, idna, six, ipaddress, pyOpenSSL, cffi, cryptography and pycparser wheels.

cp27- stands for Python 2.7, cp36- for Python 3.6;
mu- type manylinux wheels are a common choice, as they are for Pythons that store Unicode data in UCS-4 (UTF-32) format — here’s how to check it:
$ python -c "import sys; print('UCS4/UTF-32: mu-manylinux1' if sys.maxunicode > 65535 else 'UCS2/UTF-16: m-manylinux1')"

Note for Python 3: the cp34-abi3-manylinux1 cryptography's wheel can be used with any Python version>=3.4 because abi3 support multiple versions of Python3, e.g cryptography-2.5-cp34-abi3-manylinux1_x86_64.whl (2.4 MB)

Basically, wheels are ZIP archives with a specially formatted file name and the .whl extension, containing a relocatable Python package. The package can be pure-python, but also can have pre-compiled C libraries for python bindings, so it can be installed without the need to have certain system dependencies like gcc, python-dev and other C headers/libs, often required for classic .tar.gz format packages. This also allows to use exact versions of programs bundled within each wheel. The manylinux1_{x86_64,i686} wheel platform tag was adopted in PEP-513 and will work on many linux systems, including the popular desktop and server distros in common use. Expect manylinux2 tag in future!

Simply create a new directory, for example:
$ mkdir ~/wheels_dir
and copy (or move) all the downloaded packages to that directory.

Note: If you are using a virtual environment (strongly suggested) the directory should be in the site-packages directory of your venv.

No other files (except the downloaded wheels) and no subdirs there please!

Step 2 — Install

If your current pip version is below 8.1, the newer pip version has to be installed before proceeding with all other packages:
$ pip install --user --no-index wheels_dir/pip-20.2-py2.py3-none-any.whl
It will upgrade pip to handle the new wheel format and help avoid the "not a supported wheel on this platform" error.

To install all the packages at user home level:
$ pip install --user --no-index ~/wheels_dir/*
$ pip3 in Python 3

If installing in a new or existing virtualenv, omit the --user option:

$ source bin/activate
$ pip install --no-index ~/wheels_dir/*

Pip will resolve correct installation order and dependencies automagically.

Note: Unless you install in a Python virtualenv or venv, it is highly recommended to always use --user flag with pip. It then deploys python packages under your home dir in ~/.local/lib/ In fact, this option is always On by default in distro-patched pip versions provided by python3-pip and python-pip packages in recent versions of popular distros such as Ubuntu, Debian, Fedora, etc. Please try to avoid sudo pip, as using pip with root access interferes with your OS package manager subsystem (apt, yum, etc) and may affect essential OS components that depend on the distro-supplied system python.

Run $ pip freeze (or pip3 freeze in Python 3) command to check the results and ensure all packages have been installed for your Python environment.

Congratulations! Now your pip should work with PyPI, and you can try to look up something like pip search seleniumfrom the online PyPI repo.

Verify

You can see the detailed summary of your system SSL/TLS setup by querying the installed pyOpenSSL lib directly:
$ python -m OpenSSL.debug
(a ModuleNotFoundError would mean the pyOpenSSL package was not installed)

Cryptography’s linked OpenSSL shared lib doesn’t conflict in any way with your system Python’s openssl version. It may now be a good opportunity to also update your collection of root SSL certificates for the future by installing the latest python certifi package.

Why it works

Earlier versions of pip (before 10) only used the standard library’s ssl module (which is a Python API to system OpenSSL library) without any possible fallback to other libraries like cryptography. Since version 10, pip now can use pyOpenSSL with cryptography, if present in the environment.

The wheel of cryptography package includes recent OpenSSL library that supports all TLS protocols as high as v1.3 regardless of what's on your platform (PyPI expects pip to support TLSv1.2). That's why this wheel weighs 2.1 Mb -- the archive ships a shared lib binding:

$ strings site-packages/cryptography/hazmat/bindings/_openssl.so | grep OpenSSL -m1  
OpenSSL 1.1.1a  20 Nov 2018  
$ python -c "from cryptography.hazmat.backends.openssl import backend as b; print b.openssl_version_text()"  
OpenSSL 1.1.1a  20 Nov 2018  
$ python -c "from OpenSSL import SSL; print SSL.SSLeay_version(0)"  
OpenSSL 1.1.1a  20 Nov 2018  
$ python -c "import requests; print requests.get('https://www.howsmyssl.com/a/check').json()['tls_version']"  
TLS 1.3

The Cryptography wheel contains a statically-linked OpenSSL binding, which ensures that you have access to the most-recent OpenSSL releases without corrupting your system dependencies.
This will allow you to continue to use relatively old Linux distributions (such as LTS releases), while making sure you have the most recent OpenSSL available to your Python programs. (https://cryptography.io/en/latest/installation/)

In Python 2, the standard library’s ssl module began supporting PROTOCOL_TLSv1_2 flag explicitly since version 2.7.9, while in Python 3 - since version 3.4; but TLSv1.2 connections would only work if and only if the TLSv1.2-capable system-wide OpenSSL library was already available in the system by the time Python was being compiled and linked against it. TLSv1.2 requires a minimum of OpenSSL 1.0.1 to function but OpenSSL 1.0.2 (or later) is generally recommended (it uses TLSv1.2 by default).

If you do have Python 2.7.9+ or 3.4+, and its ssl module had been, in fact, compiled against system openssl, say v1.0.2k, then even old pip (such as v6.0.8) would still be working with PyPI as of the time of this writing, and you would not even need cryptography for that. To check the standard library Python ssl and system openssl versions:
$ python -c "import ssl; print(ssl.OPENSSL_VERSION)" && openssl version
OpenSSL 0.9.8o 01 Jun 2010

Even if we upgraded some outdated distro-supplied openssl, or compiled the newest one, we can’t just re-link the existing Python installation to it: the ssl module was hard-linked to the system-supplied OpenSSL upon compilation/installation of Python, and not vice versa. So, basically, one could not take advantage of new TLS protocols without recompiling/reinstalling Python itself (that should be versions 2.7.9+ / 3.4+ at least) to link it to the new system openssl library. This is where the above pyopenssl+cryptography approach comes to the rescue.

It took me a while to solve this. So, I hope I can help you in case you have the same problem.