Saving a friend with APFS Data Recovery

“Peter, can you help me with my computer?”

tl;dr — Back up your files. Don’t update your OS unless you have to. Security features make it hard to recover your own data when things go wrong.

You can read Russian translation here: https://howtorecover.me/sokhranenie-druga-s-apfs-data-recovery

It was 7 am, and I sat in the KHOP prayer room with my Bible open but my eyes shut as I drifted from sleep to wakefulness. Amber is an intern̶cessory missionary and a talented musician there, and she knew I’m always willing to try fixing broken gadgets.

“Of course!” I cheerfully replied, hoping I could quickly install some software for her and bike off to work by 8 am as usual.

“It doesn’t start up… it’s showing a folder...” Uh oh. “Tonight I’m busy visiting kids in the hospital until 9 pm, is that too late?” She had time, so we agreed to meet.

You can choose a startup disk by holding down option. On Amber’s MacBook Air though, no drives appeared — not even the recovery mode.

Target Disk Mode is a lifeline at times like this. Restarting holding down T, then plugging in her MacBookAir7,2 to my MacBookPro11,3 using a Thunderbolt wire should let me use see her computer as an external drive. That way, I can rescue her files from a volume that won’t boot. But nothing appeared in Finder.

Disk Utility could see the SSD, but no volumes. Oh dear. I used File > New Image to copy the whole drive to a file for further work.

I tried the following three commands from SuperUser, but no luck.

• hdiutil attach -ignorebadchecksums /Volumes/path/to/image
• hdiutil mount -nomount -readwrite /Volumes/path/to/image
• hdiutil convert imac.dmg -format UDTO -o output.img

Prosoft Data Rescue is an excellent tool to scan a drive for lost files. When a security guard outside a North Korean labour camp in Russia told me to delete my photos, I agreed — and recovered them afterwards using Data Rescue. I tried it on Amber’s MacBook Air, and it found the EFI system partition with some plist files, but none of her data.

I opened up the DMG using Hex Fiend, and I could see lots of data there, but no strings I recognise (e.g. JFIF for a jpg photo, ID3 for an mp3 song file).

Browsing for new ideas, I read a success story of ddrescue.

sudo port install ddrescue

sudo /opt/local/bin/ddrescue -v -n -c 4096 /dev/rdisk3s2 Rescue.dmg Rescue.log (to copy the data off)

sudo /opt/local/bin/ddrescue --force -v -c 4096 Rescue.dmg /dev/rdisk0s2 recovery.log(to copy the data onto another drive)

But even that wouldn’t mount. Scanning the drive with DiskWarrior couldn’t find her files. Exhausted, I packed up, cooked dinner, and went to bed. And as I slid peacefully into sleep…

Aha! It must be APFS!

Obviously. Why didn’t I think of this earlier? My MacBook Pro is running macOS 10.11 El Capitan. APFS volumes won’t mount on my old OS.

In September 2017, Apple released macOS 10.13 High Sierra. They decided to reformat everybody’s disk from HFS+ (the old filesystem that can be mounted on Linux, is safe to backup, and is faster to read/write) to their own proprietary APFS, missing the chance to use open-source ZFS or EXT4. What could possibly go wrong?

Amber had taken her computer home, so in the morning I couldn’t try Target Disk Mode. However, I had the DMGs from Disk Utility and ddrescue. I used DiskMaker X to make a High Sierra installer, and ran it on a spare computer.

I tried double-clicking the DMG created by Disk Utility from Finder. “no mountable file systems.”

I opened Disk Utility to try again, and the second time, I saw a marvellous sight. It was asking for an encryption password! In fact, the ddrescue DMG can also be mounted.

I asked for her password, and it worked! Her files are back.

Now everything can be copied off to a safe place, such as my external backup drive. And in future, I hope she learned to back up!

Lessons learned

• APFS drives do not mount on older versions of macOS. If you’re helping a user, you need a computer with macOS 10.13 High Sierra.
• Encryption stops Prosoft Data Rescue and DiskWarrior from helping you recover your own lost files. Security is not always a good thing.
• Back up before updating to mac OS 10.13 High Sierra.

I also think that heuristic data recovery methods used for HFS+ might not work on APFS right now. If people want, I can write some scripts to generate lots of small example APFS-formatted DMGs for research purposes.

Update: The root cause

I’ve since learned that the problem happened because of a failed software update. In the middle of installing High Sierra (and reformatting the drive), the machine appeared to hang, and the user rebooted it.

About the author

Peter Burkimsher is currently working for OSE, a memory card manufacturer in Kaohsiung, Taiwan. I fixed a Mac Plus when I was 8, iPods for friends in school, and a class set of iBooks at age 15. I studied Electronic Systems Engineering at Lancaster University and graduated with first-class honours in 2011, before going off on Working Holiday visas to see the world. Now I have 3 years continuous relevant work experience, I’m looking for a suitable job in New Zealand so I can get the Skilled Migrant Category visa. (Canada’s Express Entry and Australia are also options).