Domain discovery using Certificate Transparency logs
You just finished your initial recon of the penetration test of your new customer, but you only found a few domains. This is going to be a long night, you hoped for a bigger attack surface.
But now you remember you’ve seen a post about certificate transparency and how it will identify even the test, acceptance and hidden domains for your customer. You’ll dig through the transparency logs and bam, you’ve found a broader attack surface. This must work out.
You’re glad you’ve found these new domains, they gave you several quickly exploitable targets. These domains were never found if you had stick to the well known approaches you used before, the approaches of subdomain scraping, bruteforcing, dictionary attacks, asn discovery and permutation scanning.
Certificate Transparency logs have been around since 2013 and contain about hundred million certificates now. The logs have been introduced as a measure against incorrectly issued certificates, after the issuance of more than 500 fraudulent certificates by Diginotar while being hacked.
Ben Laurie and Adam Langley started Certificate Transparency to make the certificate issue process more transparent and visible to everyone.
Issued certificates are being appended to an ever-growing Merkle hash tree. Similar to how blockchains work. The log can be accessed by calling the API (for example https://ct.googleapis.com/rocketeer/ct/v1/get-entries?start=1&end=10) which will return the base64 encoded certificate in the merkle leaf. Most major certificate issuers append to the log and when there were 187 issued certificates found without the domain owner’s knowledge, Google Chrome required Symantec to append all certificates to the log.
Every certificate in the log contain information about the issuer, subject and all dns names the certificate is valid for. The dns names are most interesting, containing but not limited to host names for vpn, mail and web servers. Now we can retrieve host names for specified companies, countries or every other filter we can think of.
Using a tool I created a while ago (https://github.com/dutchcoders/ct/) I exported the complete log into an Elasticsearch environment. By just setting filters on a specific company (using kibana) we can retrieve all registered certificates. This will show the subdomains for given company, but also the related companies and other urls being used by the company. Subdomains for test environments are often very interesting, they contain sometimes production data, but weakly secured.
While digging around, I noticed also a lot of wildcard domains issued for large organizations and being handled by third parties. If you take the certificate of fbi.gov for example, this is a wildcard certificate but issued and validated to Cloudflare. This means that Cloudflare can setup any new fbi.gov subdomain, without their knowledge.
Another interesting url I found was Szymon.gruszecki.has.hacked.prod2.uber.com. Try brute forcing this ;-)