Launching Better Tweetdeck 3, a post-mortem
Context: Better TweetDeck is a Chrome extension I launched in 2014 to improve TweetDeck web app with a lot of features.
Better TweetDeck 3 launched at the beginning of the month! And boy it was an odd one.
Users were happy, so was I, finally shipping something I’ve been working in my spare time for more than four months now. Everything looked great! :)
Except something happened, that somehow I knew could happen but I didn’t think would actually happen: my Embed.ly bill usage exploded.
In case you might ignore, Embed.ly is an API that provides embed and display data for pretty much any link you could think of, from the Facebook post of your mom to that very specific xkcd strip you love. Their job is to take a given URL and to give you data about it so you can display a nice preview of it. And they do that very well.
I started using their service back in 2.0, the very first rewrite of Better TweetDeck! I thought back then that this time I would write an actually maintainable code (coughs).
Using them had ONE big advantage: embedding a SoundCloud song or an Instagram picture was done with the same code, meaning I could move faster and have a much cleaner codebase.
But it comes at a price, their pricing is simple: you pay for 50,000 URLs/month first, then every unique URL is something like $0.0007.
At the time, I was using it for only a few services like Bandcamp, TED, SoundCloud, and others I may forget.
But it was okay because:
- I was below the 50k URLs monthly quota!
- Embed.ly decided to sponsor me one year of their service, free of charges! And I honestly thought I could renew this sponsorship somehow, so I never worried about that.
Of course at some point the sponsoring wore off and I had to pay bills. It happened around the time when I got my current job, so I could actually pay for the service. But paying $20 a month for an extension that wasn’t generating any money was not ideal.
So I got rid of Embed.ly for all the services except TED and Bandcamp, because the former’s API was impossible to access and the latter just didn’t have an API at the time (and still doesn’t).
Fast forward to when I started working on Better TweetDeck 3.0, a huge rewrite, full of promises (pun intended).
So then I thought “Maybe I could give Embed.ly a new try again?”, after all, I was ”only” using 35,000 URLs per month out of 50,000 with two providers. So using it for more than 25 providers would be fine, right?
…Well, no. Everything went wrong.
I under-estimated the number of URLs my users would query to Embed.ly. Like, REALLY under-estimated.
In 24 hours my usage went from ~35,000 to 66,000 URLs. That’s roughly 2,750 URLs per hour and 45 URLs per minute. Keep in mind that my quota was 50,000 URLs for a month, for $20.
So yeah, of course Embed.ly warned me that if things continued at this rate, I’d be paying more than $300 per month for their service.
Here is an accurate depiction of what was going on in my brain when I got that email:
I had to act, and fast, otherwise I’d end up broke before even realizing it. So I pushed something like ~6 updates in 48 hours so Better TweetDeck would stop using Embed.ly altogether. It was annoying for everyone because users had to re-enable the extension given the frequent change of permissions scopes during the updates. Not cool.
At the same time I was explaining the situation on Better TweetDeck’s twitter account stating that if I don’t do anything about the situation, I would have awfully expensive bills. That’s where I had the idea of opening a Patreon page, so people could donate and help me to handle the cost of Embed.ly if I could not ultimately get completely rid of it.
Luckily I found a solution and Better TweetDeck doesn’t use Embed.ly anymore and if it does in the future, I’ll be sure to protect my back before doing so.
But finding the said solution took almost a week and Embed.ly continued to bill me for quite a lot of URLs. I could afford it, but I’d rather use that money on something else. Better TweetDeck doesn’t make any money, so it was like throwing money in a firepit. In the end I owe Embed.ly 134 dollars for the month of August 2016 which are covered at 70% by donations (Patreon + one-time donations on PayPal).
And for that I have to thank every single person who donated on PayPal or pledged on Patreon. Thank you, you are one awesome piece of mankind.
Okay, what happens now?
That’s a good question!
Now that Embed.ly is not an issue anymore I can focus on the cool part: making Better TweetDeck even better. Here is the “roadmap” of Better TweetDeck in the coming months (not necessarily in chronological order):
- Get the Opera port published on their store. The code is ready and submitted to them for a review.
- Successfully port Better TweetDeck on Microsoft Edge. If it works, it could be a huge boost in popularity/exposure for Better TweetDeck and that’s always nice to have.
- Work on Better TweetDeck 3.1 which will notably includes a muting engine that’s more powerful/flexible than TweetDeck’s current one and probably other things.
- Launch Better TweetDeck’s website. The extension never had a proper website, that needs to change for a couple of reasons, and I think it deserves it now.
- Improve the documentation of Better TweetDeck’s code so contributors can more easily help me improve it and make it even more awesome.
- Find a way to make “donation based” features in the extension so those who donated (on PayPal or Patreon) could get exclusive features. I don’t have any ideas for now but I think re-adding Embed.ly-powered providers for that would be cool since it could actually be reasonable this time.
I want to keep the core features of Better TweetDeck free but I also want to reward those who make the effort of giving money, even if it’s done once.
- Find a streamlined way to distribute beta versions of Better TweetDeck to patrons on Patreon so they can be involved in the development process and improve the overall stability and quality of the extension.
- Keep doing awesome stuff!
Thank you for making it this far. I know this was a weird way to start my Medium “presence” but I’ll be writing about funnier and more interesting stuff in the future :)