Separating the Notification Wheat from the Chaff
Sending carsales.com.au alerts straight to Slack with a little Zapier magic.
In explaining why I feel so strongly about ad blockers, notification spam and content marketing, I wrote a rather lengthy diatribe about what it feels like to be both an end user and front end developer in the industry. It didn’t feel right to leave this in, as it implies I was talking specifically about carsales.com.au. They are instead merely yet another notification and email in my inbox I want to automate away for good.
- I want to know when something happens
- You want to up sell me
- Your advertisers want to influence me
There are two types of notifications in this world: those that you originally wanted to see, and those that a product owner wants you to see.
It’s an ambitious but naive goal, but I want to remove as many touch points with your business as possible, and receive information and alerts on my terms. I don’t care that your real estate company now offers home loans. Besides the possible conflict of interest, I know that it’s not for my benefit, but for yours. And hey, didn’t I sign up when you had houses in my budget and in my area?
A more traditional approach to the problem would be running some Node.js scripts on CRON jobs, scraping the target website and checking for new matches, firing them off as a tweet or PushBullet notification. Indeed this is something I’ve done in past while hunting for houses:
But a couple of things are happening in this space.
Slack is the new PushBullet is the new Twitter Bot
An increasing number of colleagues and friends are turning to personal or family Slack instances. Even on a free tier you get an unlimited amount of webhooks, with each one easily personalised via display name and avatar. Slack gives you all the nice things we’ve come to take for granted like link unfurling, media embedding and the like. And that’s not to mention the enormous amount of bots and integrations that you can add on to further your personal digital assistant goals.
Scrapers are Out
Websites are also pretty watchful for web scrapers, even if only targeting the SaaS ones for now. If the key value your business provides is aggregating information, and someone can scrape that for free, it makes sense that you want to protect that at all costs.
I was never lucky enough to try Kimono before it was shut down, but sites like carsales.com.au have blocked just about every service that has popped up in their wake.
Using Email Alerts Instead
The idea I had was to instead repurpose the antiquated but still somewhat useful feature of email alerts. There are no RSS feeds. There are no mobile push notifications. We make do.
A Note on Free Tier Limitations
From what I’ve seen so far, I really like Zapier. Mistakenly, I built the first version (pictured above) using the pro user “multi step workflow” on a trial membership, and then had to pare it back. I’ll explain later on.
Set Up Your Alerts
The most obvious first step is to configure your alerts on carsales.com.au. It should be as simple as performing your search and then following a “Save Search” or “Receive Alerts” prompt. Ensure you elect to receive your notifications immediately.
Configure Your Email Trigger
In Zapier you must create a new Zap with an email based trigger. This is where Gmail really shines, due to their great search syntax and more advanced filter options. Our use case is relatively simple.
Gmail users are specifically looking for “New Email Matching Search”.
And, once connected to your specific account, we’re looking for the particular filter below. This handles the three particular situations in which there is only “1 car found”, multiple “x cars found”, and when you have a multitude of results across many saved searches “x/new cars listed in your saved searches”.
subject:”car found” OR “cars found” OR “cars listed in your” from:carsales.com.au
Email Parsers Won’t Help Here
Throughout this process I tried countless email parsers. They ranged from the simple but elegant one baked into Zapier, to the supposedly machine learning powered algorithms. They all fail a pretty simple use case, repeaters.
Email parsers are primarily geared towards pulling out key information from legacy email formats, generated in plain text from servers running in someone’s basement. Zapier makes an enormous amount of sense here, where no or low-touch solutions mean you can pump an old email format through a parser, grab out the data fields you need, and then re-send a better looking email through Campaign Monitor or Mail Chimp. Quite amazing, actually.
But the moment you have dynamic data, say three cars instead of one, these tools might as well not even bother.
Except, Zapier’s FaaS environment is locked down, with no packages besides
node-fetch to polyfill the ES
fetch function, as well as strictly disabling npm. Drat.
Filthy Regex It Is
Stack Overflow was adamant that you should never parse HTML with RegEx. So of course I did exactly that.
Once you have configured your two input fields, namely the email subject and body, you should be able to use the following code. Replace with your webhook URL obviously.
Sending to Slack
The eagle eyed amongst you may have noticed that I post the result to Slack from directly within the code here. This is because after my trial expired the pro feature of multi-step Zaps was removed. Knowing full well that webhooks can be invoked from pretty much anywhere it was simple enough to do it from our function.
In The Flesh
At some point in time between writing my paid Zapier version and my free Zapier version, I appear to have broken the link unfurler. The unfurler takes the URL, visits the site, and grabs key information like an image preview, the car location and price. Here it is when it was working.
And here it is more recently, with the unfurler remaining firmly furled.
I should probably fix that.