Practical Application of RSYNC
For High Availability Mission Critical Environments
RSYNC- copies files and folders to or from a local and or network destination and is considered the “swiss army knife” of the file replication world. The purpose of the document is to prove a practical approach to the use of rsync – everyday use. Any one of us can read a man page and this paper is not intended as such.
The following is a breakdown of the most common switches and syntax that I have found useful for performing quick backups.
Most common switches:
- -u copy only updated or changed files (must be first option)
- -ah always add this switch
- –progress gives verbose output
- –delete remove files at the destination that aren’t in the source
- –compress-level=0 provide no compression
- –inplace create files and folders respectively
Lastly, include the source and then the destination, always include the source first followed by a white space and then the destination. Here’s what a pretty basic backup command looks like:
rsync -ah –progress –delete –compress-level=0 –inplace /source /destination
Additional notes and advanced application:
Copying server to server:
(arguments always given for path in *SOURCE* -> *DESTINATION* format)
rsync -ah -e ssh –progress –delete –compress-level=0 –inplace /source email@example.com.*.*:/destination/folder
Use switch -u to copy only files that have been updated or changed, this option *MUST* be the first one declared
rsync -u -ah -e ssh –progress –delete –compress-level=0 –inplace /source firstname.lastname@example.org.*.*:/destination/folder
An extremely useful option to invoke is bwlimit which is used to limit the bandwidth use…
The number is interpreted as kb/sec and the option must the last option called before the source and the destination and entered.
rsync -u -ah -e ssh –progress –delete –compress-level=0 –inplace –bwlimit=1000 email@example.com:/home/vgizzi/origbackup/* ~
SPECIAL NOTES ON SYNCING BETWEEN
A SYNOLOGY RACKSTATION AND A FREENAS APPLIANCE
Anyone who has used a Synology Rackstation has undoubtedly become familiar with more than one or two quirks. Use the following code to get rsync copying to a FreeNAS appliance correctly. Had I made the decision originally, I would have saved the money spent on the Synology and bought two Dell PowerEdge C2100 servers and threw FreeNAS on them.
On the Synology
chmod root:root rsync
ln -s /usr/syno/bin/rsync /usr/local/bin/rsync
On the FreeNAS
chmod root:wheel rsync
ln -s /usr/local/bin/rsync /usr/syno/bin/rsync
I had to sync from the Synology to the FreeNAS because of rsync protocol data-stream errors (presumably because FreeNAS root group is set to wheel and root as an auxiliary group) I don’t believe this will be an issue when going from FreeNAS to FreeNAS as both will be set to wheel.
- -t Preserve date and time stamps
- –perms Preserve file permissions (or attempt to at least)
RSYNC in its final form!
rsync –rsync-path=/usr/syno/bin/rsync -u -a -h -e ssh -r -l -t –no-p –no-g –progress –delete –compress-level=0 –inplace –bwlimit=5000 /volume1/hdswork/* firstname.lastname@example.org:/mnt/SHARES
This script can also be used to back up local files as well