macOS Time Machine on your Cloud Storage system (OneDrive for me)

Customized Time Machine Logo with outer space background from https://www.flickr.com/photos/fhke/240086966

1st Edit — 2024–08–04

A couple of weeks ago, I noticed my mac was complaining about being unable to create new backups. I tried to mount my sparsebundle containg my Time Machine backups and lo and behold… it was corrupted. I spent a few days trying to fix it without much success.

Time machine was convenient because it was preinstalled with your mac, built by Apple and free. However it always felt off. The process for setting it up was simple but not useful to me. Customizing it to fit my needs on the other hand, was complicated, rigid and proved unreliable. So, I’ve decided to switch to Carbon Copy Cloner. I chose this because I don’t want to pay for a cloud backup service, since I already pay for OneDrive, and it seems that in CCC, you can set up any custom destination for your backups.

Original article

TLDR

If you want to use TimeMachine to be able to quickly restore your mac to a previous state, or configure a new mac if your storage drive gets damaged, or you lose your machine, then follow the instructions below. Read the full story on how and why I did this under the Instructions section.

Tested on macOS Sonoma 14.1.1, should work on most older versions. Works with any storage system you are paying for like Dropbox, OneDrive, Google Drive, iCloud etc.

Prerequisites

1. Cloud Storage system like OneDrive, Dropbox, Google Drive, iCloud etc. that has a macOS application which can automatically sync your local files to the cloud and vice versa.
2. Sufficient space or the right subscription to said cloud service

Instructions

1. Open Disk Utility on your mac (use spotlight, launcher or your fav 3rd party tool and type ‘disk utility’)
2. Click on File => New Image => Blank Image
3. Configure this new image as follows in the order written here:

• Save As => whatever name you like, I called mine TimeMachineBackups
• Name => same as what you wrote on Save As
• Image Format => sparse bundle disk image
• Size => what you think your upper limit of backup size should be. I set mine to 300 GB. This will not reserve the 300 GB in advance, but it will be your upper limit. Can be resized later with a small terminal hassle.
• Format => leave as is: APFS
• Encryption => 128-bit AES. More info on this choice in the long version of this article below.
• Partitions => leave as is: Single partition — GUID Partition Map

4. Choose the location where your new sparse bundle disk image shall be placed. I placed it on my OneDrive. I had to make sure that I set this file to “Always Keep on This Device” for best experience with OneDrive.

5. Click Save

6. Open Terminal and type the following command, then hit Enter. When prompted, enter your password:

sudo tmutil setdestination /Volumes/{mounted-disk-image}

hint: {mounted-disk-image} should be the Name you chose earlier when configuring the image. You can also see it in Finder on the left side as a new Mounted image.

Next you need to configure Time Machine.

1. Go to Apple Logo on top left => System Preferences => General => Time Machine
2. Click on your Time Machine volume with the same name as {mounted-disk-image}. Mine is called TimeMachineBackups
3. Click on Options to configure how often you want the backup to occur

Finally, you should follow this guide to quickly set up a script that will automatically mount the .sparsebundle you created earlier every time you log into your mac, otherwise you might end up without backups:

Automatically mount encrypted drive (sparsebundle) on Login on macOS

Optional but HIGHLY recommended

In order for TimeMachine not to back up everything in your system and create insanely large backup files, you need to configure an excellent “Exclusion List”. Otherwise, you’ll be spending double the storage space both on your mac and on the cloud storage service. You do this by adding entries into the ‘Exclude from backups’ table you saw in step 3 above.

Hint: Use finder and sort your files and folders by Size. You will find all kinds of things to exclude.

Mine contains over 80 entries. You need to press cmd+shift+. to see some of these. Some of the most important ones are:

1. All large applications like any games, Logic Pro X, Final Cut Pro, Xcode, anything you can easily install manually later.
2. All of your Cloud Storage files. You’re gonna have to find these. Mine were in: ~/Library/Group Containers/UBF8T346G9.OneDriveStandaloneSuite and also in ~/Library/Cloud Storage
3. All games stored in places like ~/Library/Application Support/Steam, ~/Library/Application Support/Blizzard etc.
4. ~/Downloads
5. /bin
6. /Library/Caches
7. ~/Library/Caches
8. /sbin
9. /System
10. /usr
11. /tmp
12. /etc

After all of this, my first backup was about 120 GBs in size. Time Machine is clever enough to only add the changes on top of that first backup every time it runs a new one, just like Git. It is now quick to update, and I feel safe knowing that I’ll be able to get a lot of my settings, utility applications and config back if something goes wrong with my machine.

And now, here’s the longer story behind how and why I made some of these decisions…

The Long Part

Why save Time Machine backups to the cloud

I have only recently started using a Mac, and coming from Windows & Linux, I noticed a lot of small but useful features missing. One of the features that I depend on regularly is the ability to place a window on the left half of my screen and another on the right. I found out from my Mac-savvy friends that there is a small app called Magnet that can help with that. I bought it immediately and it solved my problem well enough. I then started digging a bit more on how to optimise my mac and found this channel called Snazzy Labs on YouTube. These two videos helped me get a lot of really nice tools that I now depend on day to day to work efficiently:

Feel free to skip to the Terminal Tips using the timestamp in the description of the video below.

Useful apps video

Nobody is paying me to promote these tools or the Snazzy Labs YouTube channel, I’m just sharing my experience. I now depend on Alfred, Bartender, Magnet, Dropzone, Hotkey and a bunch of other small Mac utility applications that make my life easier. I also would love not to have to go through all those steps again, and remember any customizations I may have made myself if I have to move to a new Mac, or something happens to my machine.

That’s why I started exploring Time Machine on the cloud. I already have 1 TB available on OneDrive via the Microsoft Office 365 family subscription, so I went with what I had.

The Lack of a Complete Guide Online

I’m pretty used to typing out what I need to do in a concise way into Google and getting to a useful guide with very few clicks. Wasn’t the case this time. I was getting bits and pieces from StackOverflow, SuperUser, Quora, Reddit, Apple’s Support Forum etc. but nothing complete that showed me how to solve this problem. I then tried Bing Chat with GPT-4, which didn’t really help much. It kept leading me back to the same resources and doing bad summaries of the content that didn’t really work. This was one of the many moments recently where I realized how foolish I was being when attempting to jump on the hype-train and use GPT as a search engine. To be clear, I do find plenty of practical uses for it, but not for something like this.

I then realized that I needed to roll up my sleeves and figure this out by putting all the pieces together. After a couple of hours of digging, I was happy with the result, so I decided to write this article to show y’all how to do it too.

The AES Encryption rabbit hole

If you didn’t know this already, please pay attention: ALWAYS ENCRYPT YOUR BACKUPS. It is too hard to predict what kind of stuff on your computer can end up in a backup file, since you are working with exclusions instead of explicitly including files you want to back up. It is even harder to imagine how a future attacker is going to attack you or your backup files and what they will look for inside. Keep your privates private people!

I tried 256-bit AES, thinking it would be the ultimate choice in security when backing up. It gave me the impression that it was too slow. Benchmarks needed, would love some contributions by someone who has more insight on this.

I then recreated the ‘sparse bundle disk image’ with 128-bit AES encryption. It ran faster, but I was worried that the encryption wasn’t secure enough, even though macOS itself recommends it. To find out, I started Googling. I came across this amazing answer from Lery on crypto.stackexchange.com, in which he references Bruce Schneier’s book “Applied Cryptography” where he claims that you couldn’t brute force an AES 256-bit key even if you ran a Dyson Sphere at full capacity just for this operation. If you didn’t know about Dyson Spheres, you’re in for a treat… A Dyson Sphere is a theoretical hollow sphere made out of solar panels that completely encloses the Sun and captures 100% of the energy it emits. Check out the Wikipedia article for some fun times.

Is 128-bit security still considered strong in 2020, within the context of both ECC Asym & Sym…

You could use a Dyson Sphere to brute force 128-bit AES though. But yeah… I ain’t planning on protecting my backups from a future Sci-Fi level civilisation with the luxury of using the energy of the Sun for a while just to break my key.

All kidding aside though, please take security around your backups extremely seriously, and if 128-bit AES is ever cracked then it is best to either delete your old backups or re-encrypt them.

The Exclusion List

This part was pretty annoying as it required a lot of manual work. I had to manually try to find all kinds of Caches, Applications I knew I didn’t need to recover etc. I did this by sorting files and folders by size in Finder and looking around. I also found an answer on Quora which suggested adding a few interesting System and Hidden files and folders I missed:

https://www.quora.com/What-should-you-exclude-from-Time-Machine-backup/answer/Frederick-Northrop?ch=10&oid=20701632&share=c0c57b67&srid=kTiR&target_type=answer

After getting to 80 entries, I decided to find a way to export this list so that I can recover it easily if I messed something up. Here’s how to do it:

defaults read /Library/Preferences/com.apple.TimeMachine SkipPaths

Thanks to Gilby for that:

How to export Time Machine exclusions or list them in Terminal

Cue the looney toons ending music.

That’s all folks!