Recently I became a victim of a bug that has been in Google’s code for so long that if you had started a degree program in Computer Science back when this bug was introduced, you could have graduated by now and gotten a QA job at Google where you might have the honor of fixing it.
Let’s back up a bit though. I’m the proud owner of 850GB of data. 20 years worth of photos, videos, emails and other bric-a-brac. I’ve been backing it all up for years on a handful of drives, and that works well enough.
But any good backup strategy includes off-site redundancy. There are lots of options for cloud backup these days. SaaS encrypted backup services like Mozy, while easy to use, get very expensive fast. So, a few months ago I purchased some great backup software called Arq, which encrypts and backs up your data to any of the major low-cost cloud storage services.
I went with Google Drive, which offers 1TB for $10 per month. Gradually I uploaded about 280GB to it before Amazon Cloud Drive became available as a better option for the long term, at $5 per month for unlimited storage, vs. Google’s pricing which jumps to $100 per month once you exceed 1TB.
So I made the switch, but in order to downgrade my Google Drive account, which is attached to a larger account with a decade of other Google data in it, I had to remove all of my backup files from it. Otherwise they would keep charging me $10 a month.
This’s where the trouble started.
All of my data was stored in a single Google Drive folder called “Arq Backup Data,” with around 300,000 files in it. So I dropped that folder in Google Drive’s Trash. This didn’t free up any storage space, because the Trash counts toward your quota.
Then I found the cleverly hidden “Empty Trash” button, and the Trash now claimed that it was empty:
But there was a problem. My total storage stayed right at 280GB.
“Well, it’s probably just cached,” I thought. I came back a few minutes later, and it had dropped by 0.5GB. Meanwhile, the Trash was full again. I emptied it again. The total storage dropped a little more.
OK, this is starting to get annoying.
Maybe I just have to empty the trash 500 times?
Thankfully, emptyTrash is an endpoint on the Drive REST API. It took me about 10 minutes to write a Ruby script that would hit the endpoint every few minutes. My plan was to let that run for a while and I’d be done.
At this point in the story, we have left the realm of what an Average User would ever do. But I just wanted the problem solved quickly.
I started running my script, and after a while the storage figure just didn’t drop any further. And the trash looked empty. It wasn’t working.
Google Drive has a Quota page that shows which files count toward your quota. All of my files had ended up here, “orphaned” in the sense that they don’t appear anywhere else in Drive:
So I dragged 20 files from the Quota page to the Trash just to see what would happen.
And it worked. I was able to delete 20 files.
I’d now invested a couple hours and I’d found two bugs: Trashing “lots of stuff” breaks the Trash, and after that, at some point, all of those files become orphaned.
Using the web interface, I found I could move about 100 of the orphaned files from the Quota to the Trash at a time. When I tried to trash more than 100 at a time, the request failed:
So I had a solution. I could delete about 0.036%, of my files at a time. At about 10 seconds for each pass, this would take around 8 hours of tedious mouse work.
My other options at this point were:
- Ask Google for help
- Write another script to find and programmatically remove the orphans
- Hire a neighborhood kid to to do the mouse work?
- Record a fragile mouse macro, and have it run overnight?
- Dispute the credit card charge?
Disputing the credit card charge would be too risky, actually. If they shut down my account, I would lose a decade worth of Google data.
Google has no phone support for people who only pay $10 a month. So I went to the web forum for Google Drive and proclaimed that I had discovered these bugs, hoping for some assistance.
A day later, a cute penguin popped up with an official response from Google. It boiled down to “Try clearing your cookies.”
I’ve learned to contain my excitement when a technical conversation turns to cookies. “Clear your cookies” is one of the oldest tricks in the technical support manual — an easy way to blame a user for anything, and it solves problems slightly less often than never. There will be no delicious chocolate chip payoff.
Also, why have we tainted such a wonderful dessert?
I replied, telling Google that the problem was on their end. That their advice was not going to change that. It had only made me hungry.
But I was just another anonymous user. Another Cookie Monster. And I was pretty sure from the salutation on the original canned response (“Best of luck!”) that I would never hear from them again. It’s been 5 days, and I haven’t heard back.
With a sigh and with my other options looking dubious, I started coding. I wrote a little Google App Script to find and delete the orphaned files.
I wanted to create a workaround that other people could use. There are hundreds of complaints about storage quotas and Trash issues in the product forum, dating back to 2012. Many of them point to this bug and have the same canned response. Given that Google Drive launched in 2012, my hunch is that the Trash has never actually worked.
So I perfected my App Script, returned to the product forum, and posted the link to my script on a couple of the tickets.
I finally had a fix, and it was generalized. My script deletes about 500 files at a time. I set it up to run on a schedule, every 10 minutes. That worked great! For about 2 hours. Then I received this:
And that was it. Rock bottom. All of my attempts at a nice automated fix had failed. Today I’m running my script manually every few minutes while I work. 500 files at a time.
Google Drive has roughly five features:
- Add stuff
- Star stuff
- Share stuff
- $$$ Upgrade Storage $$$
- Remove stuff
But Google has smartly followed the 80/20 rule, and only 80% of these features work. I’m kidding of course — the 80/20 rule isn’t an excuse to leave 20% of your product broken — but I am left with some questions:
Would Google ever choose to cash in on some of their brand value to keep users perpetually locked in via a bug they will never fix? While this bug may be profitable for Google, I don’t think it’s a profit-driven bug. I think it’s just a really long-standing bug.
If this bug is indeed profitable and also very hard to fix, where will the incentive come from to fix it?
Will it come from Google not being evil? (aka preserving brand value)
Will it come from an engineer’s pride in the well-oiled machine?
Will it come from a valiant technical support penguin who won’t rest until the problem is solved?
What if Google’s approach to user support is is to rely on the 1% of experts among it’s billion users to solve problems for everyone else, including Google?
What if the product were “Comcast Drive”? Would you use it?
Comcast Drive kind of sounds like a place to store porn, doesn’t it?
Will Google ever become more like Comcast?
How long is “ever,” anyway?
Finally, will Google send me some chocolate chip cookies for my effort?
Bugs like this have always been super frustrating. In the old days of boxed software, you weren’t forced to make Herculean efforts to avoid being charged every month. But today we have Software as a Service. Once you are locked into a SaaS subscription, you have almost no bargaining chips when issues arise.