Real-time synchronization of files without SSH support with LSYNCD and RSYNCD

A basic discussion of file storage and synchronization

Web site development to a certain stage, sometimes need to be uploaded by the server files to focus, here to discuss if the focus of the problem, as if distributed load balancing, CDN acceleration, multi-host, multi-domain blablabla, not discussed.

Centrally shared scenarios (in the case of CentOS 6.5 for example) the common practice is file synchronization, shared hard disk space, hardware stacking, and distributed file storage of four.

1. File synchronization is to use the tool timing or automatic synchronization of scattered files to a centralized storage area, such as cron with FTP, SSH, rsync and other implementation of timing synchronization;

2. The practice of sharing space is to use the file storage protocol for mounting and management, such as Mount, Samba, NFS;

3. Hardware stacking is implemented using large disk space, such as arrays;

4. The last one is clustered and distributed file systems, such as Taobao file system TFS;

The 2nd is not very stable, the 3rd high cost, the 4th type of implementation complex, the simplest practical is the 1th species.

The 1th option is to synchronize with the Rsync command, which combines SSH credentials with an authorization-free login, which has several negative effects:

• Need to add a local account

• Not real-time synchronization

• Configuration slightly verbose

Well, we want to realize the function of real-time monitoring, real-time synchronization, simple management, easy maintenance, here are introduced LSYNCD and RSYNCD two commands to achieve centralized synchronization of the file management. LSYNCD is responsible for monitoring file changes, initiating file push, RSYNCD is responsible for receiving synchronous instruction, authenticating and writing files to local.

Configuration and management of LSYNCD and RSYNCDConfiguration of the receiving end RSYNCD

General Linux servers are installed with the rsync command, without installing one.

Yum Install rsync

Configure as Daemon run mode

Vi/etc/rsyncd.conf

Lock file =/var/run/rsync.locklog File =/var/log/rsyncd.logpid file =/var/run/rsyncd.pid[img] path = changed to local write folder absolute path Comment = Special Offer image files uid = root gid = root Read only = no list = yes auth users = rsynccli Ent Secrets file =/etc/rsyncd.secrets hosts allow = 192.168.100.0/255.255.255.0 192.168.200.234

NOTE: [img] is the virtual directory name, which is used in the initiator configuration, the hosts allow for the IP address sent to the originator, AUTH users for the authenticated user name, and the password exists in the/etc/rsyncd.secrets file.

Vi/etc/rsyncd.secrets

Rsyncclient:password

Note: This password file is slightly different from the password file that was sent in the originator, with one more user name.

Start RSYNCD (assuming the current root environment):

/usr/bin/rsync--daemon

Add this command to/etc/rc.local for automatic start-up.

See if the Service log has errors

Tail-f/var/log/rsyncd.log

Configuration of initiator-side LSYNCD

Installation

Yum Install LSYNCD

If prompted not to find lsyncd you need to download Epel.repo to/etc/yum.repos.d/directory and install it again.

Vi/etc/lsyncd.conf

Paste the following code into this file

settings {    logfile =  "/var/log/lsyncd/lsyncd.log",     statusFile =  "/var/log/lsyncd/lsyncd-status.log",     statusinterval =  10}---------------------------------- sync files to product  Server-------------------------------sync {    default.rsync,     Source= "Replace the absolute path of the folder you want to monitor and sync,"     target= "rsync://[email protected][to receive end Ip]:/img",     delete = false,-- doesn ' t delete files on the  Remote host eventho they ' Re deleted at the source. this might  be beneficial for some not for others    delay =  5, -- We want to delay the syncing for 5 seconds  So we queue up the events    maxprocesses = 4, -- we only want to  use a maximum of 4 rsync processes at same time     rsync = {        password_file =  "/etc/ Rsyncd.secrets_master ",        compress = true,         acls = true,         verbose = true,        owner = true,         group = true,         perms = true    }}

Important: Note the above delete = False parameter, if you do not delete the receiving end of the file, set to False, otherwise it will automatically delete the initiator without the file on the receiving side!! Please choose carefully, generally set to false.

The above configuration assumes that the synchronization account is called Rsyncclient, password files stored in/etc/rsyncd.secret_master, note: Here the account is not a local account, just RSYNCD virtual account.

The receive-side virtual path is called IMG, which defines the configuration of the reference receiver, which is just a virtual directory, which is actually pointed to in the configuration file.

Vi/etc/rsyncd.secrets_master

Password

Note: There is only one plaintext password for this file line.

Finally, start the LSYNCD service

Service LSYNCD Start

Check log to see if it starts normally

Tail-f/var/log/lsyncd/lsyncd.log

If this line indicates that the normal startup and configuration is successful.

Normal:finished a list after exitcode:0

Set LSYNCD to start automatically

Chkconfig--level 3 LSYNCD on

If all goes well, the initiator has monitored the file changes and automatically pushes to the receiving end after a 5-second delay, and you can test the actual results yourself. again, please practice multiple times in the test environment and observe at least one week before you configure to the production environment!

??Summarize??

The advantage of this scheme is real-time synchronization, without SSH protocol support, no need to add local account, configuration is simple, deployment flexibility.

< Zhu Yu [email protected]>? ?

Real-time synchronization of files without SSH support with LSYNCD and RSYNCD