esync: synchronize but keep conflicts

Often, I have to backup quickly my files to an hard disk when I have to travel and then clean my machine before. To remove any sensitive data (anything protected by a NDA, or things I want to keep for me like passwords). Obviously I could just rely on the daily, weekly and monthly backups I do, but I don’t really trust them. So I backup to a another disk.

At then end my user folder is available in different versions on each disks with often duplicates files never edited. Of course I never remove them when I am back from travel, reinstalling the machine from scratch, using time machine, or sometimes from that HD since timemachine from time to time just fail. With the results that along the years I accumulated a lot of hard disks in my office. I thought it was about time to solve that situation. So I started to write esync with the following goals:

  • Merge different folders from different sources in one
  • Keep the conflicted file side by side in the folder
  • Copy should be enough fast, possibly done in a concurrent manner

Here is the first version of esync:

Usage is very simple:

$ ./esync ~/Desktop/backup_t1/benoitc /Volumes/T1/benoitc 
building file list …
done (51834 files processed)
copy files … done
size: 1584453344 copied: 51834 conflicts: 1858

It is written in Erlang so I was able to prototype the script very fast and do most of the copy and conflict checking concurrently. Conflicts are stored by naming the conflicted file “FILENAME.conflict-TIMESTAMP”. This file is stored in the same folder. On each sync a log is kept in your folder.

For now the script does nothing much. I may add more feature in the future like some configuration to fix the path where logs are stored and some other stuff. Will see. Anyway I hope this script may be useful for you.