Google Drive and Takeout: A Backup-Sync Odyssey

A few weeks ago, I came across a tool from Google I hadn’t encountered before, Google Takeout. Takeout allows you to download a backup of your Google data. You can get much of the data you generate when using Google including your Gmail inbox and the data I was interested in, Google Drive. I believe in having multiple backups and while Google Drive can function in this manner, it’s never bad to have backups of your backup.

I go to Takeout and first request a backup of all my Google data. Then realizing that this particular data set might have a lot of data I wasn’t looking for at the time, I requested another, smaller backup. After both requests, Takeout informed me that it would send an email notification when each backup was ready.

By then, it was late. I went to sleep and didn’t think about it again until a few days later when I turn my laptop and notice Google Drive isn’t syncing files I’m putting in it.

“I can’t be over quota,” I protested to myself. “I have 100GB of space in there. Something’s wrong.”

No.

And yes.

I looked through my files in Windows Explorer. All told, I had parked about 17GB in the Google Drive folder. I’d synced all subfolders. If all of the folders contained 17GB of files, then how was Drive reporting both online and in the sync app in Windows, that I had over 100GB of files in Drive?

I noticed a folder in the online, browser based version of Drive that I hadn’t seen before: Takeout. It was full of .zip files. Since several days had passed since I’d requested the backup, I’d forgotten the backup requests I’d placed.

I delete the folder. It doesn’t help. In fact, the folder seems to come right back almost as soon as I delete it. I’m still over quota.

Since I’m again doing this late at night, I’m getting desperate for a solution and not exactly thinking straight. I get the fantastic idea to begin deleting items from Drive and seeing if that helps to bring me back under quota. But before I delete anything, I decide I’m going to make a backup to ensure that I can get all of my files back. That’s what you’re supposed to do. That’s what you train yourself to do when you work in IT.

I copy my Google Drive folder to my Documents folder. I’m ready, I think. I plug in an external hard drive and move every file and folder in Google Drive to it. “Cool,” I think. “I have a backup and now everything should clear up.”

Nothing cleared up.

The Takeout folder was still there. I delete it online. I empty the trash. I’m still over quota. I have no clue what’s going on. I make a post in the Google product forums. I didn’t get a quick response, so I figure I’ll go to sleep and see what’s popped up by morning. If anything, Google would delete the files out of my account if I remained over quota, I figured.

I put the files back into my Google Drive folder in Windows. It doesn’t sync, but that wasn’t exactly unexpected. But I couldn’t let it go before going to sleep, I find a thread in the product forums where people were having similar issues to mine. The thread contained a link to a form you could fill out to inform Google you were having the issue and they would empty your Drive folder. I fill it out and go to sleep.

Next day, my quota’s gone down and Takeout folder is gone, but there’s another problem, a larger problem: my documents aren’t syncing. The documents I’ve created online and placed back into the folder from my Windows based backup, are not syncing.

I click on a few of the files in Windows and after Chrome launches, it informs me each time that the link the file was pointing to, was gone.

My in-progress play.

My in progress book.

Blog posts, poems, essays, none of them are coming up.

I make another post in the Google product forums. A helpful user tells me I should chat with Google Drive support. That they might be able to recover some files if it hasn’t been too long.

When I get over to support, the gentleman helping me agrees that this is the case. He says he’ll start recovering, but makes no promises about the number or selection of files he can restore. At this point, it doesn’t matter. I’d be grateful receiving anything back. He starts the recovery and I go to sleep (I do a lot of computing at night).

The next day, there are files again in my Drive. The Takeout folder is back, but by then, I’d looked up what it was — the backup I’d requested and completely forgotten. And yes, the notification was in my email that it was ready to retrieve. The overall problem was that I’d made the assumption that the notification would come and I’d go to a page to download the archive instead of them shoving it into Drive. Obviously, making things worse was that since time had passed, I’d forgotten about the request in the first place.

I download the Takeout backup to an external hard drive before embarking back through my browser to Google Drive support because again, I was up near quota.

This time, the specialist, the same one as before, spends a ton of time with me trying to figure what’s going on. My trash doesn’t seem to empty. He has me deleting all kinds of files. Some files that I need. Ultimately, he finds that I have a ton of orphaned files in my Drive online. Some of them are the online versions of documents that I’d had in my folder in Windows but became orphaned when I pulled them out of the Google Drive folder. Still, he has me deleting so many files. When I see the names of some of them, I want to take my iPad and bang myself over the head with it. Writing assignments. My play again. My book again. Saving them at this point was impossible.

By the time we’re done, I’m close to the right amount of files in my Drive, sans a great deal of documents that I couldn’t really stand to lose.

Late again, I decide to just go to sleep and survey the full extent of the damage later. But before I go to sleep, I again remember the Takeout backup (I really had a bad time remembering my Takeout backup). Surely, it had to have at least some of my files, I figure.

I open up each and every zip file in the repository. Each zip seems to have a version of each folder. The folder I want is the one where I keep my writing.

And as it turns out, there is a recent backup. I look through it. Yes!!! My book is there. My play is there. So many of the files I’d deleted twice, were there. And the placeholder documents I’d had in the backup I’d created, were not there. Instead, there were actual .doc files. I combine folders and files from my Takeout backup with the one I’d created in Windows and combine that with the ones that synced down from the newly-restored files in Drive online. My play and other files made it back online and are synced again. I’m still working with getting the whole Drive back in order, but I’m pretty much back where I was when I requested the backup in the first place.

I’d somehow been given a second chance by the recovery tool. If it hadn’t snatched back the Takeout backup from the jaws of being overwritten, I wouldn’t have been able to save so much of what I’d worked on. The gentleman at Google Drive had been helpful throughout and he’d saved me a great deal of heartache. And some embarrassment. Not all, but some.

Lessons Learned:

1. If you’re a Windows user, remember that those .gdoc files that show up in your synced files are NOT document files the same as Microsoft Word files. You may already have known this and if you did, then kudos to you. If you did not know that, again, the .gdoc files that show up in your Google Drive folder are NOT document files. These files are merely pointers to the online documents that you created in Google Docs/Drive. Do not EVER move these files out of the Google Drive folder. That will delete the corresponding document in Google Drive, thus orphaning your document. Even copying them to another drive renders them useless in that target drive. If you do move these files, you may or may not be able to chat with Google Drive support to recover your file, but only if you do not wait too long to do so. I chatted with Google Drive pretty soon after deleting my files and I was not able to recover everything, which leads to …
2. Make regular backups of your data kept in Google Drive. This may seem counterintuitive, if you see Google Drive as a backup. However, Google Drive can function in ways that can be defined as both sync and backup. If you are placing copies of files into Google Drive and turning off the sync so that the files reside in the cloud only and you are not regularly “touching” them, then you’re using Google Cloud in a backup manner. If you have files that you move into your Google Drive folder expecting an exact copy to be made in the cloud, while at the same time, keeping and perhaps working from the local copy, you are using sync. It may seem a minor distinction and many may consider Google Drive to not be a backup product (Google itself is coming out soon with a new or reengineered service it is explicitly selling as backup, using Google Drive) but when your files are synced with Windows and the local Windows copy goes bye-bye for whatever reason, the cloud version goes as well. This is not an issue if you have turned sync off for any specific folders as there is no local version to control the online version of that file. Given that, it’s important, especially if you’re using Google Drive to sync your data, especially Google Docs files that don’t have a corresponding true document locally (i.e. you made the Google Docs on a Chromebook and haven’t saved .doc versions of those docs somewhere), to regularly generate and download a backup of your files from Google. Google Takeout will give you .zip archives of your files inside of your Google Drive as previously described. The cool thing is that Takeout will give you documents instead of .gdoc files. Use the 3–2–1 backup protocol to decide where to put those files. I’m keeping copies on external hard drives as well as inside of another cloud service I’m using in a backup manner. Yes, I combined all the .zip archives and password protected them before sending to the other service. If you have decent Internet at home and some money to spend on it, I suggest getting a Synology network storage drive and storing it there and/or a pure backup service. In fact, you could have your Google Drive files sync directly to your network drive as well, but don’t forget you’ll still get .gdoc and not Word documents, so still do a backup. Store it somewhere offsite. And while you’re at it, remember to keep adequate space in your Google Drive for your Takeout backup and don’t forget that you requested it in the first place.
3. Do not go moving Google Drive files and folders around in Windows willy-nilly. If you’re moving within your Google Drive folder, that’s okay. But if you’re moving files to and from it and have Google-made documents there, you’re asking for trouble. Best to download the .doc or .xls from Google, first, before you go moving any files around. Files you don’t regularly need, just unsync the folder with Windows and leave them in the cloud. You can always download them from there again.
4. If you run into problems with Google Drive, the Google product forums for Drive seem to be pretty good. Google Drive support is really good. The support person I talked to those couple of times helped me get a bunch of files back, including the Takeout archive back that I used to rebuild my document cache.
5. Nobody’s perfect. I’ve been doing IT for nine years at my current job and since I was 17 and I still made mistakes that contributed to the stress of those few days. I did not do with the pros do, what we do at work. But I’ve learned my lessons and will be more mindful in the future of my own data practices. If you’re not a pro, do what the pros would do, just not our mistakes.

Takeaways:

1. Regularly back up your Google Data using Takeout. Add this to your 3–2–1 backup protocol. If all you have is a Chromebook, get a high enough capacity flash drive to make your backup.
2. If you’re syncing Google Drive files to your local machine, still manage your files in Google Drive online, as much as possible.
3. As often as you can, download Google documents, as you create them, as Word documents or one of the other file formats that Google offers.
4. Visit Google Drive product forum or support as soon as you run into any problems. There is a time limit to their file recovery abilities.

Originally published at christopher a. kess.