Rsync to the rescue of SCP

Did you ever try to download a big file over Internet? I have to do it sometimes… For example, to copy a database dump. These kind of files can be over 1Gb.

I used to use the following command to copy a file using SCP:

$ scp user@server:/a/path/to/a_big_dump.tar my/local/path/.
a_big_dump.tar 42 % 612MB 443.2KB/s — stalled

Unfortunately, this command was failing most of the time and the connection was lost if I had a network hiccup, or if my laptop took a nap.

Lately, I tried to download a big dump from my work office. I never managed to fetch it. I tried 5 times, I prayed every God I already heard about… Nope… No way to grab this dump...

Hopefully, I found another command: rsync. This command is really helpful because it can resume a download in case a transfer was partially complete:

$ rsync --rsh='ssh' \
--progress \
--partial \
user@server:/a/path/to/a_big_dump.tar \
my/local/path/.
receiving file list ...
1 file to consider
a_big_dump.tar
2293760 0% 1.06MB/s 0:07:06

I can quickly explain the different options:

  • --rsh : specify which remote shell connection we want to use. In our case SSH.
  • --progress : display the current progress of the transfer
  • --partial : allows to resume a transfer instead of restarting it from the beginning.

Originally published at kevin.disneur.me on November 27, 2014.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.