We started using Firebase’s realtime database back in November of 2015. It was one of the biggest mistakes I’ve ever made building our product. Literally.
At first, we loved it. We needed something to power the messages for “Codey”, the educational coding bot in our online IDE. Firebase was easy to integrate, the documentation was clear, and the pricing was reasonable. We had it up and running in a commit or two.
Then came the first major outage.
Having customers blow up your inbox because the product you’ve spent years perfecting is down is not fun. It’s unbearable when it’s caused by something outside of your control.
In the 3 years since starting to use Firebase we’ve suffered from so many outages I’ve literally lost count.
Someone is keeping count though, here’s Firebase’s page on StatusGator:
Granted Firebase has multiple services and StatusGator doesn’t let you filter to just the realtime database, but for comparison, here’s the page for Appsignal, another service we use built by a small team in the Netherlands:
We’re also a small team of 3 developers (up until this summer it was just yours truly) yet our uptime over the past 6 weeks (as far back as we can go on Stackdriver) is 99.992%. We’ve rarely fallen below 99.9% and we’re damn proud of that.
Firebase is a service acquired and backed by Google Cloud (someone once told me the acquisition price was $80MM, but I cannot find evidence of that online). Yet their uptime is a fraction of ours and the many other services we depend on.
We’ve never gotten an answer to this and the fact we’ve stuck with them for so long is on us. I would never have written this based on my continuing to subjugate myself and our customers to a shit product.
But they just did something that really got me going.
Yesterday, Firebase’s realtime database was down for two hours (during which major parts of our application were 👎). On their status page it showed a big ol’ red line on their real-time database. We alerted our customers via our status page and monitored until it had been addressed.
StatusGator also tracked this outage:
Yet if you go to Firebase’s status page today, this is what you will see:
A single little “service disruption” dot. No big red line. No major service outage.
This is unacceptable.
When I tell our customers something is wrong outside of our control, I already feel bad for passing the blame. But to see the company who caused the downtime try to cover it up is… nauseating.
Downtime happens. In the case of Firebase, you might say that “uptime happens.” But when your product breaks, no matter what, transparency is a MUST. Status pages are used to make product decisions and I’m sure we’re not the only one that redirect our customers to it when there are issues.
Needless to say, we’ve been looking for an alternative to Firebase and are nearing our breaking point. So far we’ve identified Google Cloud’s Firestore which we like because we run atop GCP but fear we’re in for a similar surprise from them…
So here’s my question to you: what have you used for realtime data Self-hosted solutions? Different SaaS offerings?
Please, tell me.
If someone from Firebase reads this, feel free to correct me if you feel I’ve miscategorized how you’ve handled this.