Fixing Exception Message in Bug that I Found in check-webkit-style Script (release 0.4 for DPS911)

Introduction

Before fixing previous bug, I acctually experienced big problems with running one of the scripts.

When I was preparing to my first contributions to WebKit I had some issues with running “check-webkit-style” script. I was getting the exception coming from Tools/Scripts/webkitpy/common/system/autoinstall.py (line 360).

Exception message:

“Could not download Python modules from URL “None”.
Make sure you are connected to the internet.
You must be connected to the internet when
downloading needed modules for the first time.”

It was trying to install the pep8 module but was failing because it did not know what URL to download it from and said that I was not connected to the internet. But actually, I was connected to the Internet and had a stable connection. Then I reinstalled Python and WebKit developer tools, but nothing helped.

After that, I reached out to the WebKit community on IRC and we began to look where the problem can be. The first thing that came to mind is blocking of TLS < 1.2 by pypi (files.pythonhosted.org) and some contributors suggested me to reinstall Python manually, but it did not help as well. Then I reached out to other contributors who helped me before with other bugs concerning building WebKit and they did not know as well what can cause it, but helped me with pinging Jonathan Bedard (from Tools/Operations team, who owns most of the code in Tools/Scripts).

Jonathan reached me out on IRC webkit channel and we began to try to find where the problem can be. Firstly he suggested me to delete Tools/Scripts/webkitpy/thirdparty/autoinstalled and run the script again to see what is going to happen, the folder appeared again and the same error was shown. Jonathan told that he saw this error on High Sierra machines, but never Mojave(!!!). After we continued to look where the error can be and Jonathan came up with idea that probably I have an old version of urllib2 ssl libraries, and I uninstalled them and installed again, but it did not help as well. We tried reinstalling multiple libraries and were wondering which library actually is responsible for my error. After trying all the libraries we could imagine the error still existed.

After some time Jonathan came up with small repro case:

import urllib2
url = ‘https://files.pythonhosted.org/packages/source/p/pep8/pep8-0.5.0.tar.gz'
urllib2.urlopen(url, timeout=30)

I ran it on my machine, while Jonathan ran it on lab machines in Apple and surprisingly we got different errors:

Lab machines got this:

<urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert 
protocol version (_ssl.c:590)>

And I got this:

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify 
failed (_ssl.c:727)>

That was weird because I had the first one before I reinstalled python manually.

Then Jonathan came up with the idea that Python on my Mac does not have access to its system certs so it can’t verify any of the URLs. After this, I researched some errors again and one of them was very weird, it said:

“The directory ‘/Users/appletest/Library/Caches/pip’ or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo’s -H flag.Collecting urllib2-ext”

It was trying to reach an appletest user on Mac which was the “super admin user”, but I deleted it 2 years ago and somehow it was still on my machine. After running “sudo pip install requests” I got a very similar error with appletest user:

“The directory ‘/Users/appletest/Library/Caches/pip/http’ or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo’s -H flag.The directory ‘/Users/appletest/Library/Caches/pip’ or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo’s -H flag.”

Finally, we came up with the conclusion that Python is owned by appletest user (which was deleted, I was running user called — yevseytsev). We wanted to delete it but we saw that my Mac has a weird set up, and I could not even delete the appletest user under users folder or in settings of my Mac, so I decided to completely reset my Mac and reinstalled the system from the very beginning. After this, everything was working good and I was able to run any webkit script on my Mac.

Going back to the exception message that I received on the beginning:

“Could not download Python modules from URL “None”.
Make sure you are connected to the internet.
You must be connected to the internet when
downloading needed modules for the first time”

It can be easily seen that not only the internet connection can cause this problem, it also can be caused by wrong set up of your system, users or Python.

I thought the exception message should have a more detailed description and say about other possible causes as well (system/user set up or Python), in order to protect future contributors who can get this error from wrong interpretation and understanding what can cause it.

Filling Bug on Bugzilla, Fixing It and Creating Patch

I filled this bug on Bugzilla and began working on the code contribution.

https://bugs.webkit.org/show_bug.cgi?id=197046
https://bugs.webkit.org/show_bug.cgi?id=197046

I improved the exception message with the following text and created the patch:

“…Additionally, make sure Python is owned by the current user and it can access its system certificates in order to verify the URLs.”

Feedback of Community

After I submitted my patch I received a feedback that there is something wrong with my patch because it is too big and affects lots of the files.

Feedback about the first patch

I researched this and understood what I was doing wrong. I was working with the WebKit’s snapshot of the tree but I needed to work with the git version. I checked out a master branch of the WebKit, added my fixes again and created a new patch.

Changed files are now the right files
Patch passes all the internal tests on the Bugzilla
Current status on Bugzilla

Status as of 11 pm, 18 April 2019 — discussing improvement with community in order to get the final version of the new exception message.