Any day that you see one of your projects make it to public release is a good day, and today is a damn good day. Today I introduce the world to CertStream — a free service and simple libraries for getting data from the Certificate Transparency Log (CTL) network in real time. This allows anyone to write extremely simple code (or even a bash script) to react to SSL certificates being issued, as they’re issued.

If you’re impatient like me, feel free to skip the whys and the hows of this article and just go to https://certstream.calidog.io/ …

I’m a huge fan of open source software. It, in a very organic way, creates a community of individuals working together to build something awesome not for monetary reward, but because they’re passionate about it. Sure tools like GitHub & friends have their problems, but It’s truly breathtaking to see how much of a catalyst for social communities they’ve turned into. It’s the ecosystem and collaborative tooling they offer that allows so many amazing projects to flourish in the way they do. As with everything though, for some communities they find it’s not always one-size-fits-all.

Enter Frida

I’ve long chattered away to anyone who would listen about Frida, as it’s by far my personal favorite tool for RE tasks. It’s an amazing piece of open source software, extremely well designed, and the applications for it are almost limitless in their possibilities. It’s also written largely by one person, Ole André Vadla Ravnås — whom I would consider a giant in the RE community and a good friend. He has been tirelessly working on it for at least the past 5 years, and is excited about it every single time I’ve talked to him, seemingly without fail. He’s also one of the most humble engineers I’ve ever had the pleasure of collaborating with, and someone I truly strive to emulate on a daily basis. …

In part 1 we examined HOW to parse Certificate Transparency Log information using Python, but the real value comes from being able to retrieve, store, and search them cheaply and efficiently, which what I set out to do.

NOTE: If you’re coming from the pentester.io article, all you have to do is email me for access to this dataset — ryan@calidog.io !

Lets harvest some certificates!

According to the Certificate Transparency RFC, the api endpoint get-entries is what we’re looking for to pull down the entries from each certificate transparency log. Unfortunately for us it uses limit/offset pagination (in the form of the start and end parameters), and a lot of the running logs only let you pull down 64 at a time. Thankfully the Google CTLS, which make up the majority of the certificates, all use a max response size of 1024. …

This is part 1 of a series I’m doing on collecting, parsing, storing, and querying 250,000,000+ certificates from CTL logs, You can find part two here!

While building our soon-to-be-released first product — phishfinder, I spent a large amount of time thinking about the anatomy of a phishing attack, and the data sources that would allow us to detect evidence and artifacts of phishing campaigns as they were getting started, before they have time to do any real damage.

Among the sources we’ve integrated (and definitely one of the cooler sources that exists) is the Certificate Transparency Log (CTL), a project started by Ben Laurie and Adam Langley at Google. At a high level, a CTL is pretty much what it sounds like — a log for storing a cryptographically-verifiable immutable list of issued certificates from a central authority, stored in a Merkle Tree. …

This is truly an exciting (and terrifying!) time in my life.

Today I introduce the world to Cali Dog Security: a software company with its sights focused on making no-BS security products that emphasize practicality, a beautiful user experience, and making security tools affordable for anyone who needs them. The first problem we’re tackling is phishing, but much more on that later.

We’re bootstrapped with no VC involvement whatsoever, and we plan on keeping it that way. That means our customers - and only our customers - are the ones we answer to.

The creed of Cali Dog

Our core mission is simple - make security tools ubiquitous and practical. …

I’ve worn many hats over the years - everything from IT support and DevOps to starting the internal application security team at a large multi-national defense contractor. Seeing all sorts of different infrastructures and their security problems has lead me to a few truths that I hold about every organization, regardless of its size:

• If you are an obstacle, people will step around you. If you make it a battle you will lose. Period. There are always going to be more of your users than there will be of you and your team, and no amount of firewall rules is going to change that. Your users will go to ridiculous lengths to circumvent you if they feel it necessary to get their stuff done, and you simply can’t be everywhere at once. …

Hi there, my name is Ryan, and I’m an engineer.

I build lots of stuff, including WSStat and maintaining Sulley, and have a true drive and passion for security. I spend my days either breaking the internet and doing security research, or helping to make security tools more accessible to the everyone with my company - Cali Dog Security. You can find more about me and my past on my website, or by just shooting me an email (my username [at] gmail.com) .

I’m a firm believer that security tools that exist today are woefully inadequate for the threats they’re supposed to mitigate (for the most part), and that there is much more that can be done to change how we combat security threats in a modern environment. …