Linux uses rsync to sync directory _linux

This article describes the problem of using rsync to synchronize two machine directories under Linux. After using Rsync synchronization, you can keep the directory consistent (with delete operations).

Data synchronization method

1, pull data from the host

The process that is started on the standby machine

Sync command:

Rsync-avzp--delete Root@{remotehost}:{remotedir} {Localdir}

Parameter description:

• -a parameter, equivalent to-rlptgod (-R is recursive-L is a linked file, meaning copy link file;-p means to keep the file original permissions;-T to maintain the original time of the file;-G to maintain the original user group of the file;-O to maintain the original owner;
• -Z compression when transmitting;
• -P transmission progress;
• -V information such as the progress of the transmission;

Example:

Rsync-avzp--delete root@192.168.1.100:/tmp/rtest1/tmp/

2. Push data to standby machine

Processes initiated on the host

Sync command:

Rsync-avzp--delete {localdir} Root@{remotehost}:{remotedir}

Example:

Rsync-avzp--delete/tmp/rtest1 root@192.168.1.101:/tmp/

Automatic sync Configuration

Describes the method for configuring a password without entering the synchronization.

1, the use of SSH key

This method can be used to synchronize directly with the rsync command without entering a password during synchronization.

Generate SSH key on host:

SSH-KEYGEN-T RSA

Add PubKey to standby machine

Ssh-copy-id-i ~/.ssh/id_rsa.pub root@192.168.1.101

or manually add:

Execute the following command on the host to get PubKey:

Cat ~/.ssh/id_rsa.pub

Add key content to the standby:

VI ~/.ssh/authorized_keys

Use Pexpect to enter passwords automatically

The sample code is as follows:

#!/usr/bin/env python
#-*-coding:utf-8-*-

import pexpect
import time
import Traceback

def Dorsync (user,passwd,ip,srcdir,dstdir,timeout=3600):
  cmd = "Rsync-azpq--delete {srcdir} {Ruser}@{rhost}:{dstdir } ". Format (
    ruser = User,rhost=ip,srcdir=srcdir,dstdir=dstdir
  )
  try:
    ssh = pexpect.spawn (cmd, timeout=timeout)
    print cmd
    i = ssh.expect ([' Password: ', ' Continue connecting (yes/no)?], timeout=5)
    If i = = 0:
      ssh.sendline (passwd)
    elif i = = 1:
      ssh.sendline (' yes ')
      ssh.expect (' Password: ')
      Ssh.sendline (passwd)
    ssh.read ()
    ssh.close ()
  except:
    #print traceback.format_exc ()
    Pass

if __name__ = = ' __main__ ':
  dorsync ("root", "123456", "192.168.1.101", "/tmp/rtest1", "/tmp")

This is the code implemented using Python, which can be implemented in other languages as appropriate.

Other

1. How will rsync be killed in the course of execution?

It is safe to kill an rsync process and run the whole thing again; It'll continue where it left. It May is a little inefficient, particularly if you haven ' t passed--partial (included in-p), because rsync'll check AL L The files again and process the file it is interrupted on from scratch.

Rsync is safe to kill and will work properly the next time it starts.

2, rsync can not specify the time period;

1 The problem can be solved by kill

2) or using Pexpect's timeout parameters to control

3 You can first filter out the name of the folder through find, and then use rsync to sync this can be based on the characteristics of existing business, such as:

Find/tmp-name ' * '-newermt ' 2016-03-08 '! -NEWERMT ' 2016-03-20 '

3, rsync in the process of writing files synchronization (such as the recording process to perform rsync operation)

After testing, rsync synchronizes some of the file's content, and then when the file is written, then rsync keeps the file consistent.

4, when the number of files to reach more than millions, rsync synchronization scan changes in the file is time-consuming

The above is the entire content of this article, I hope to help you learn, but also hope that we support the cloud habitat community.