linux parallel with rsync

Jerry’s Notes
What’s next?
Published in
1 min readOct 23, 2019

使用多線程的方式去加速資料複製!

應用場景

複製大量檔案到遠端的NFS server (Amazon EFS)

### rsync
$ sudo time rsync -aR /usr /shared/a
7.56user 12.57system 38:11.98elapsed 0%CPU (0avgtext+0avgdata 25528maxresident)k
1015784inputs+1769448outputs (0major+13178minor)pagefaults 0swaps

### rsync + parallel
$ cd /usr; time find -L -type f | sudo parallel — gnu -j 8 -v rsync -aR {} /shared/b
real 13m38.729s
user 2m49.075s
sys 1m25.342s

### Monitor
$ sudo iftop -P
$ watch -n 1 “sudo netstat -tunp | grep 2049”
$ watch -n 1 “ps -ef | grep rsync”

--

--

Jerry’s Notes
What’s next?

An cloud support engineer focus on troubleshooting with customer reported issue ,and cloud solution architecture.