rsync - swapping the direction (client/server) - push/pull

So, I've always run rsync in one direction (as most people do) Server A -> Server B (backup server). I now have a situation where I need to reverse this as some other scripts need to run before and after the rsync job. Can you simple change the running of rsync so it's running on server b instead?

So, the current set up is:

Runing on Server A:

rsync -avz --progress /local/server/path/ user@serverb:/path/to/storage

Can I change that to:

Running on Server B:

rsync -avz --progress user@serverb:/path/to/storage /local/server/path/

Am I correct in presuming the above swap will have no effect on the data being stored on Server B and that rsync will run from Server B, connect to server A and pull the data over?

Seems like a simple question but I've been working on a few bash/backup scripts recently so my brain is a bit... fried and I need fresh eyes!

Yes, you are correct. Just reverse the paths.

Just wanted a second opinion - I got the fear no matter how much I read and experimented!

I think we've all been there...after a while the brain just kind of raises the white flag!

MartinD said:

So, I've always run rsync in one direction (as most people do) Server A -> Server B (backup server). 

Click to expand...

I've never done this.  I always pull from the backup server.  That way if Server A is compromised, they can't just nuke the backups.  It also allows me to schedule all backups in one crontab instead of many.  

Downside is you can't do something like "Do mysqldumps and then run rsync" on the clients.  The rsync is going to come in at a fixed time.  The ways around this are:

1. Just assume all mysql backups will be done by time X and backup then

2. Have your mysqldump wrapper script create a tag file and have your backup server rsync and check if the tag exists

3. Have your mysqldump wrapper script make a notation in some sort of cloud database like Amazon SimpleDB or whatever

4. A proper multi-host batch management system, which is probably extreme overkill

I run rsync out of xinetd on the backup client.  My rsyncd looks like this:


max connections = 1
log file = /var/log/rsync.log
timeout = 300
[root]
comment = hostname root
path = /
read only = yes
list = yes
uid = root
gid = root (or wheel)
hosts allow = w.x.y.z,a.b.c.d
exclude from = /etc/rsync_exclude.txt
secrets file = /etc/rsync_secrets.txt
auth users = backup_user1,backup_user2

Hence only users with the correct passwords from certain hosts can connect.  I also limit who can connect in iptables.

And then in /etc/rsync_secrets.txt (which must be mode 600):


backup_user1: password
backup_user2: password

Plain text, unfortunately, but I use a unique password for each backup server/client combo.

More or less my common exclude file:


/var/tmp
/var/lib/mysql
/var/mysql
/tmp
/dev
/proc
/sys
/etc/udev
/home/virtfs

This assumes you're doing a backup my MySQL and storing it somewhere.

That is the Gospel of Rsync Backups According to Raindog.  

MartinD said:

Just wanted a second opinion - I got the fear no matter how much I read and experimented!

Click to expand...

Is that the "is this command going to nuke my directory even though I've checked the flags 5,000 times" fear?

If so...

Thanks!

Yep, I know what you mean however, even after doing the dry-run there's still a 'what if' especially when the data is sensitive. I have taken backups of the backup.. but there's the hassle factor involved in restoring that if things do go wrong.

I think as tonyg said it's just my brain raising the flag and me needing some fresh eyes for reassurance!

Running on Server B:

rsync -avz --progress user@serverb:/path/to/storage /local/server/path/

Click to expand...

Shouldn't it be like this?

rsync -avz --progress user@serverA:/path/to/storage /local/server/path/

I know you knew it and was a copy/paste typo, anyhow we people would point out those.

I got the fear no matter how much I read and experimented!

Click to expand...

You are on "Anxiety Disorder". GAD?

raindog308 said:

Downside is you can't do something like "Do mysqldumps and then run rsync" on the clients.  The rsync is going to come in at a fixed time. 

Click to expand...

I've had no issues with writing scripts on the backup server to run an SSH command remotely (say, mysqldump), prior to running the rsync command. You can even check return status (ssh exits with the exit status of the remote command or with 255 if an error occurred), to ensure the mysqldump ran without issue prior to starting the rsync.

Alternatively, I've begun running mysqldump piped into a compressed zip, piped over SSH, such that the files are never written to disk on the remote server.

Example: 

ssh $SSH_HOST -l $SSH_USER "MYSQL_PWD=$MYSQL_PASS mysqldump -h $MYSQL_HOST -P $MYSQL_PORT -u $MYSQL_USER $MYSQL_DB | gzip" > "$BACKUP_DEST"

Click to expand...

Pulling from backup server removes risk of attacker deleting backups but means encrypting the backups is not as useful. I guess you could a) write API thing to take the backup and transfer it to slave or b) push to one backup server and have a second backup server pull from the first

rsync -avz --progress user@serverb:/path/to/storage /local/server/path/

Click to expand...

I use this for B to A syncing...

rsync -avz --delete --bwlimit=xxxxxx --progress -e "ssh -p xxxxx" user@serverb:/path/to/storage /local/server/path/

Mid said:

You are on "Anxiety Disorder". GAD?

Click to expand...

lol - no, not at all. It's a term of phrase like a figure of speech