All You Need to Know About Lets Encrypt CAA Bugs_Part 1
On February 28 2020, Lets Encrypt announced that it was revoking approximately 2.6% of their active TLS certificates.
This translated to 3 millions certificates that needed to be revoked by March 4 2020, hopefully without breaking the internet. Affected users were notified via mail to renew their certificates before the revocation date. Lets Encrypt additionally worked with partners to renew as many certs as possible before the March 4th 2020 deadline. More than 1.7 million affected certificates were replaced in less than 48 hours(wow). However, due to the fact that more than 1 million certificates had not being fixed by the deadline, the revocation was halted, to avoid breaking the internet. On the flip side, Lets Encrypt certificate are valid for only 90 days meaning that at some point, organizations will have to renew them either way.
Although Lets Encrypt notified users, not all users were notified due to factors like wrong email address, spam filters, technical debt might have contributed to some hosts not being fixed.
Note that the CAA bug can be fixed by renewing your certificates.
Renewal procedures will vary depending on how they were setup. The following link will provide pointer on this.
Wondering what happened?
Consider a certificate request containing N domain names that needs CAA rechecking; Lets Encrypt Certificate Authority software (Boulder) picks one domain name and checks it N times. In practice, this means that if a subscriber validated a domain name at time X, and the CAA records for that domain at time X allowed Let’s Encrypt issuance, that subscriber would be able to issue a certificate containing that domain name until X+30 days, even if someone later installed CAA records on that domain name that prohibit issuance by Let’s Encrypt.
Fun fact that i learned from Patrick Gray:
When Lets Encrypt validates your domain, it does checks on it from different servers/viewpoints to ensure that the view or fingerprint is consistent from various views. This ensures and safeguards against any hijacking of domain as both viewpoints should have the same view.
From what i can gather, the Lets Encrypt revocation measure was more of a compliance obligations than an address to security risks.
So what happens when a certificate is revoked?
Once an LE certificate is revoked, Let’s Encrypt will publish that revocation information through the Online Certificate Status Protocol (OCSP). There are basically two methods for checking if a certificate is revoked: certificate revocation lists (CRLs) and OCSP. CRLs are long lists of serial numbers that have been revoked while OCSP only deals with a single certificate. The take away here is that the CA should be able to answer CRL or OCSP queries. If a CA goes down then it could take out huge sections of the web. Because of this, clients read browsers have historically been forgiving of an unavailable CA. Browsers will thus check OCSP to see whether they should trust a certificate. Note that OCSP has some fundamental problems, so not all browsers will do this check. Basically, depending on the browser, anyone visiting a site with a revoked certificate might be notified that the sites certificate cant be trusted. Something similar to the image below:
Based on my checks and assessment, CRL checking is faster when compared to OCSP. This becomes evident when you push all ccTLD as i did.
Whats Next Now:
The main question and i am of this medium was to help organizations check if they are affected. Before you throw an organization under the bus, consider that some organization outsource website management and a majority have technical debts.
Special thanks to Scott Helme who spend time to crawl and identify all affected sites and cataloged all the affected domains on his github.
Thanks to Hanno Böck for coming up with a script to help check for affected domains.
Based on the work of these gentlemen, in the next post, i will share my python scripts that can be used to check all domain affected. I’ll additionally explore ways of incorporating both works. The aim of this is to empower the community to take up the initiative of informing organizations in their home areas.
