Server Load balancer Data Synchronization implementation

Today, the teacher talked about the server architecture. Although simple, there are still some difficulties. We need to expand the architecture on this basis.

There is data synchronization during Server Load balancer. Baidu later saw the previous generation share:

 

Inotify + Rsync is used to implement batch synchronization and batch update of Linux Files in batches. This should be a frequent task for O & M personnel. Rsync is a very powerful synchronization tool in Linux, the differential synchronization method is used to upload only different parts of the file/folder and compress the uploaded parts first. Therefore, the transmission efficiency of rsync is very high, but rsync also has disadvantages, the biggest problem is that each rsync command will traverse the target directory. When there are not many files, this is no problem. Once the number of files reaches a certain scale, each traversal will consume a lot of resources, if you only synchronize directories that are frequently changed, you can ignore this problem. If there are many directories that are changed each time, inotify is required, inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism. Since 2.6.13, the Linux kernel has added support for inotify, inotify can be used to monitor various events in the file system, such as adding, deleting, modifying, and moving. However, inotify only provides the C language interface, which is not convenient to call. Therefore, we need to install inotify-tools first.

I. system environment:
CentOS_5.7-x86_64
Update source server: 192.168.9.227
Target Server: 192.168.9.226 192.168.9.228 ......

Ii. Target server configuration: 192.168.9.226 192.168.9.228 (rsync server ):
1. Check whether Rsync is installed

1. Rpm-Qa | grep rsync

Run the following command to install the SDK:

1. Yum-y install rsync

2. Define the rsync configuration file/etc/rsyncd. conf.
192.168.9.226:

1. Cat>/etc/rsyncd. conf <EOF
2. Uid = nobody
3. Gid = nobody
4. Use chroot = No
5. Max connections = 100
6. Timeout = 600
7. PID file =/var/run/rsyncd. PID
8. Lock file =/var/run/rsyncd. Lock
9. Log File =/var/log/rsyncd. Log
10. [Web1]
11. Path =/data/www1/
12. Ignore errors
13. Read Only = No
14. List = No
15. Hosts allow = 192.168.9.0/255.255.255.0
16. Auth users = www1
17. Secrets file =/etc/www1.pwd
18. EOF

192.168.9.228:

1. Cat>/etc/rsyncd. conf <EOF
2. Uid = nobody
3. Gid = nobody
4. Use chroot = No
5. Max connections = 100
6. Timeout = 600
7. PID file =/var/run/rsyncd. PID
8. Lock file =/var/run/rsyncd. Lock
9. Log File =/var/log/rsyncd. Log
10. [Web2]
11. Path =/data/www2/
12. Ignore errors
13. Read Only = No
14. List = No
15. Hosts allow = 192.168.9.0/255.255.255.0
16. Auth users = www2
17. Secrets file =/etc/www2.pwd
18. EOF

3. Explanation of the rsyncd. conf configuration file
Uid = Nobody // the user who runs the rsync daemon
Gid = Nobody // the group that runs the rsync daemon
Use chroot = 0 // do not use chroot
Max connections = 0 // maximum number of connections, 0 is unlimited
Port = 873 // default port 873

The following files are automatically generated after the rsync service is installed.
PID file =/var/run/rsyncd. PID // storage location of the PID File
Lock file =/var/run/rsync. Lock // lock file storage location. Specify the lock file that supports the max connections parameter. The default value is/var/run/rsyncd. Lock.
Log File =/var/log/rsyncd. Log // location where the log file is stored

Timeout = 300
This option overwrites the specified IP timeout. this option ensures that the rsync server will not always wait for a crashed client. timeout is measured in seconds. 0 indicates that no timeout is defined. This is also the default value. an ideal number for anonymous rsync servers is 600.

Log format = % T % A % m % F % B
With this option, you can use transfer logging to customize the fields of log files. The format is a string containing format delimiters. The format delimiters can be used as follows:
% H remote host name
% A remote IP Address
% L file length characters
% P process ID of the rsync session
% O operation type: "send" or "Recv"
% F file name
% P module path
% M Module name
% T current time
% U authenticated user name (null when anonymous)
% B actual transmitted bytes
% C when a file is sent, this field records the file's verification code
The default log format is "% o % H [% A] % m (% u) % F % L". Generally, "% T [% P]" is added to the header of each line. in the source code, a Perl script program named rsyncstats is released to collect statistics on log files in this format.

# Transfer logging = Yes
The rsync server uses FTP files to record the download and upload operations in its own separate log.

Syslog facility = local3
Specifies the message level when rsync sends a log message to syslog. Common Message levels are: uth, authpriv, Cron, daemon, FTP, Kern, LPR, mail, news, security, sys-log, user, uucp, local0, local1, local2, local3, local4, local5, local6, and local7. the default value is daemon.

Module Parameters
[Web1] // The Authentication Module name. You must specify
Path =/data/www1 // the directory to which the image is to be created. It is indispensable!
Comment = backup web // comments of this module
Ignore errors // ignore irrelevant Io errors
Read Only = Yes // This option sets whether to allow users to upload files. if the value is true, any upload request will fail. If the value is false and the server directory read/write permission permits the upload operation. the default value is true.
List = No // column files are not allowed
Auth users = Bak // The authenticated user name. If this row is not used, it indicates that the user is anonymous and has nothing to do with the system.
This option specifies a list of usernames separated by spaces or commas. Only these users can connect to this module. the user here has nothing to do with the System user. if "auth users" is set, the connection request sent by the client to this module will be sent by rsync to challenged for authentication. The challenge/response authentication protocol is used here. the user's name and password are stored in plaintext in the file specified by the "Secrets file" option. by default, the module can be connected without a password (that is, the anonymous mode ).

Secrets file =/etc/www1.pwd // table of password and user name comparison. The password file is generated by yourself.
This option specifies a file containing a defined user name: Password pair. this file is valid only when "auth users" is defined. each row of the file contains a pair of username: passwd. generally, the password should not exceed 8 characters. the default secures file name does not exist. You must specify a limit (for example,/etc/www1.pwd ). note: The permission for this file must be 600, otherwise the client will not be able to connect to the server.

Hosts allow = 192.168.9.0/255.255.255.0 // allow host or network segment
This option specifies which IP customers are allowed to connect to the module. The customer mode definition can be in the following format:
Single IP address, for example, 192.168.9.227
The entire network segment, such as 192.168.9.0/24, can also be 192.168.9.0/255.255.255.0
Multiple IP addresses or CIDR blocks must be separated by spaces. "*" indicates all IP addresses. By default, all hosts are allowed to connect.

Hosts deny = 0.0.0.0/0 // disable host

4. Create an authentication file/etc/www1.pwd, which must be consistent with the specified file name in the configuration file.
The format is Username: password, which is a security issue. We do not recommend that you use the root user in actual use.
192.168.9.226:

1. Echo "www1: 741852">/etc/www1.pwd

192.168.9.228:

1. Echo "www2: 951753">/etc/www2.pwd

In addition, we need to set the permission for this file to 600.

1. Chmod 600/etc/www1.pwd
2. Chmod 600/etc/www2.pwd
3. Chmod 600/etc/rsyncd. conf

5. Create a motd file (optional)
Rsyncd. motd records the welcome information of the rsync service. You can enter any text information, such:

1. Echo "welcome to use the rsync services! ">/Var/rsyncd. motd

6. Start rsync

1. /Usr/bin/rsync -- daemon
2.  
3. Echo "/usr/bin/rsync -- daemon">/etc/rc. Local

3. Update the source server configuration: 192.168.9.227 (rsync client)
1. inotify can monitor the following file system events:
In_access: The file is accessed.
In_modify, the file is written
In_attrib: file attributes are modified, such as chmod, chown, and touch.
In_close_write, writable file closed
In_close_nowrite: the file cannot be written.
In_open, the file is open
In_moved_from: The file is removed, such as mV.
In_moved_to: The file is moved, such as mV and CP.
In_create, create a new file
In_delete: The file is deleted, such as RM.
In_delete_self: indicates that an executable file is deleted when it is executed.
In_move_self, self-moving, that is, an executable file moves itself during execution
In_unmount, the host file system is umount
In_close, the file is closed, equivalent to (in_close_write | in_close_nowrite)
In_move: The file is moved, equivalent to (in_moved_from | in_moved_to)
Note: The files mentioned above also include directories.

2. Install inotify-Tools
Before installing inotify-tools, make sure that your Linux kernel has been 2.6.13 and the config_inotify option is enabled during compilation. You can also run the following command to check whether

1. Ls/proc/sys/fs/inotify

Max_queued_events, max_user_instances, and max_user_watches are supported.

1. Wget http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
2. Tar xvf inotify-tools-3.14.tar.gz
3. CD inotify-tools-3.14
4. ./Configure
5. Make; make install

3. Write the rsync monitoring script
 

1. VI/root/rsync. Sh

-M, that is, -- monitor, indicates that the event listening state is always maintained.

1. #! /Bin/bash
2. Host1 = 192.168.9.226
3. Host2 = 192.168.9.228
4. Src =/data/www/
5. DES1 = web1
6. Des2 = web2
7. User1 = www1
8. User2 = www2
9. /Usr/local/bin/inotifywait-mrq -- timefmt '% d/% m/% Y % H: % m' -- format' % T % w % F '\
10. -E modify, delete, create, attrib \
11. $ {SRC }\
12. | While Read File
13. Do
14. Rsync-vzrtopg -- delete -- Progress $ {SRC }$ {user1 }@$ {host1 }::$ {DES1} -- password-file =/etc/www1.pwd &&
15. Rsync-vzrtopg -- delete -- Progress $ {SRC }$ {user2 }@$ {host2 }::$ {des2} -- password-file =/etc/www2.pwd &&
16. Echo "$ {files} was rsynced">/tmp/rsync. log 2> & 1
17. Echo "---------------------------------------------------------------------------"
18. Done

-R, that is, -- Recursive, indicates the recursive query directory.
-Q, that is, -- quiet, indicates that the monitoring event is printed.
-E, that is, -- event. You can use this parameter to specify the event to be monitored. Common events include modify, delete, create, and attrib.
-- Timefmt: output format of the specified time
-- Format: Specifies the detailed information of the variable file.

Create an authentication file (the rsync client authentication file only uses the password)

1. Echo "741852">/etc/www1.pwd
2. Echo "951753">/etc/www2.pwd
3. Chmod 600/etc/www1.pwd
4. Chmod 600/etc/www2.pwd

1. /Bin/sh-N/root/rsync. Sh // check the syntax
2. Chmod + x/root/rsync. Sh
3. Nohup sh/root/rsync. Sh &
4. Echo "nohup sh/root/rsync. Sh &">/etc/rc. Local

Iv. synchronization test
Create a new file on the update source server and run the following command to check whether the file can be properly synchronized and whether an error is reported.

1. Rsync-vzrtopg -- delete -- Progress/data/www1/[email protected]: web1 -- password-file =/etc/www1.pwd

Submit the files to be updated to the update source server. In this way, the update files are synchronized to all target servers in batches through inotify + rsync, which is convenient and quick.

 

[This article is from the "Listen to the future" blog. Be sure to keep the source http://kerry.blog.51cto.com/172631/734087]