CertWatcher — Automating Certificate Transparency OSINT with Apps Script

If you read any of my last two posts, then you are aware I have been spending some time playing around in Google Apps Scripts. So, after the end of Maveristmas, I got to thinking what other fun things could I do with Apps Scripts. I am a fan of OSINT and had an idea to try and use Apps Scripts to automate, in pseudo real-time, the OSINT from Certificate Transparency. Be forewarned, what I am going to talk about here is not a new OSINT technique and there are similar scripts in python and stuff out there. However, I will walk through it in case the reader is not aware of the technique and explain my fun little project I created — CertWatcher for OSINT.

What is Certificate Transparency?

Back in July of 2011, there was the Hack of the Certificate Authority, DigiNotar (the org that creates, manages, and authorizes certificates across the Internet). This hack plus others highlighted many concerns around Certificate authorities but one of those concerns was surrounding the transparency of certificates and how to better detect the issuing of malicious certificates. So the idea of Certificate Transparency was established.

Certificate Transparency (CT) is an ecosystem designed to make the issuance of certificates transparent and verifiable. The goal is to mitigate the misuse of certificates by providing publicly auditable, append-only, trusted logs of all issued certificates. This system allows Certificate Authorities (CA) to provide all certificates to these running logs so that anyone can monitor and validate that no malicious certificates are created. Once Google announced in 2018 that the Chrome Browser will require certificates to be in the Certificate Transparency ecosystem, this pushed CT into full adoption moving forward.

In a nutshell this is how CT works (taken from the great site https://certificate.transparency.dev):

• A user requests a certificate from a CA
• CA creates a precertificate and sends it to some of the CT logs that exist.
• CT Logs send back a Signed Certificate Timestamp (SCT) to the CA proving that it has been added to the CT
• CA adds the SCT into the precertificate to make the real certificate
• The certificate is sent to the user for public use
• Throughout this, monitors may check and validate that the certificate was created and not malicious

Image of how Certificate Transparency works (Image: https://certificate.transparency.dev/howctworks/)

Example of the www.maveris.com certificate with SCT

So What?

All certificates are sent to these public logs, that anyone can monitor. Check out https://crt.sh This is the point, so as an Attacker or Defender you can monitor these logs. As a Defender, you could watch these logs for any new entries to help discover someone looking to create a new site or for any malicious certificates being created. As an Attacker, if I am targeting an organization, I can use this to get a list of subdomains for the organization (even internal systems) or watch for newly created systems that may not be as secure yet. If you bug bounty, you know that you want to hit a system to find the bug before others.

Tool: OSINT CertWatcher in Apps Script

A tool for the lazy OSINTer that focuses on discovery of subdomains related to a specific domain. The tool will run for free in Google Apps script and runs daily and returns results to a Slack channel.

This tool was designed to allow me to keep an eye on a specific domain and to get a near real-time notification each day on new domains that hit the CT log the previous day. Again, you will notice this script can be modified easily for however you want. It’s really just a simple skeleton. The reader could easily modify it to email a list, add it to a running list to keep track of over time, post to an another service besides Slack, etc. There are other things I have considered doing to upgrade the script or add features. I will take suggestions or leave that up to the reader.

How it works.

The Apps Script is set to make a request out every day. The request fetches the json data results from Crt.sh for a specific domain, excluding expired and deduplication. The returned json data is parsed for anything that was added the previous day. I did the previous day, to ensure that I didn’t miss anything if it ran at 10am but something was added at 10:05. So I accept that I will be about a day behind (near real-time). The entity name is then pulled and I create a slack payload and post it to a Slack webhook that I have created for our team.

You can find the most current code (with any new updates) on my GitHub here https://github.com/ruddawg26/CertWatcher

And TA-DA, get to sit back, drink sweet tea and get notified when an organization decides to create a new certificate for a system.

Example of how the CertWatcher could look in a Slack Post

Website: www.maveris.com

Email: info@maveris.com

Maveris exists to help your organization reach its fullest potential by providing thought leadership in IT and cyber so you can connect fearlessly. To learn more visit us at: www.maveris.com