SSL certificate abyss in openstack
I got an issue on our lab installation of Openstack: keystoneclient rejects our own certificate.
It is a normal certificate, signed by our home CA (created for lab purposes). CA root certificate is installed in trusted certificates in all related systems (it is listed in /etc/ssl/ca-certificates.crt). Curl respects it. Wget respects it. But nova/keystone (and all other openstack clients) reject it:
AuthorizationFailure: Authorization Failed: SSL exception connecting to https://our-lab.example.com:5000/v2.0/tokens [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
I checked that our CA is trusted few times. No luck. After few attempts I pinpointed problem to this code snippet:
>>> import requests
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
I done some research and found thing I didn’t want to find: http://docs.python-requests.org/en/master/user/advanced/?highlight=ssl#ssl-cert-verification
It says that requests use own CA database. It doesn’t says it, but implies that it’s ignoring system CA database and give a fuck to system administrators on managing trustiness of servers. If I decide that I no longer trust Chinese CA, than ‘requests’ will trust them on my behave regardless of my opinion.
Thank you very much. You’ve just made my life a tiny-tini bit more miserable.
How openstack deal with this? There is environment variable to control which directory to use: OS_CACERT
So you need manually explicitly say openstack client to trust your system certificates:
OS_CACERT=/etc/ssl/certs/ keystone token-get
Brilliant decision! Applause!
After some thoughts I decided to raise a bug in launchpad: https://bugs.launchpad.net/python-openstackclient/+bug/1634861