Multi-server file synchronization implementation

One, the rsync feature can be mirrored to save the entire directory tree and file system. It is easy to keep the original file permissions, time, soft and hard links and so on. Install without special permissions. Fast: When you synchronize for the first time, rsync copies the entire content, but only the modified files are transferred the next time. Rsync can compress and decompress in the process of transmitting data, so it can use less bandwidth. Security: You can use SCP, SSH and other means to transfer files, of course, you can also through a direct socket connection. Support anonymous transmission, to facilitate the site mirror image. Rsync Common Options-A contains-rtplgod (usually used-A is Enough)-R Synchronize the directory to add, similar to CP-r Option-v visualization that tells you what files were copied, how long it took, how many bytes were copied, the speed, etc., to display some information when synchronizing, let us know the process of synchronization-L Keep soft connection, copy language directory has soft connection file, use to another directory, add-l will copy the soft link itself into the directory.-When the option is added, the source files are synchronized when the soft link is synchronized-p Keep The file's Permissions property-O Keep the owner of the file, this is WWW, copy the other side is WWW, if you do not, display the number-G Keep the genus of the files, the root group, and the same in another place-D Keep device file information-T hold the file's time attribute--Delete deletes files in dest (destination directory) that do not have src (source directory )--exclude filters The specified file, such as-Exclude (can troubleshoot some directories or files, such as log files, save space) "Logs" will be the file name contains logs files or directories filtered out, different steps-P shows the synchronization process, such as rate, ratio-v More details-U is a shorthand for update, plus this option, if the file in Dest (target) is newer than SRC (source file), then the sync-Z Transmission Compression (automatic compression before transmission, after transmission, automatic decompression) Two, mode one: Using the SSH channel synchronization instructions:1, File Source: A (server side)2, Target side: B (client), C, D ..... (Receive file)1Install the rsync installation on target side B.-y uninstall yum remove rsync-y start rsync--daemon Kill process Pkill rsync view Port 873 Check to see if 873 port monitoring is turned on lsof-I.:8732Install yum Install rsync on file source side a installation-y uninstall yum remove rsync-y Key Login Configuration The first step, generate the key on the source side [[email protected]~]# ssh-keygen-t RSA//specifies that the algorithm is RSAGenerating Public/PrivateRSA key pair. Enter fileinchWhich to save the key (/ROOT/.SSH/ID_RSA)://Save key full pathEnter Passphrase (empty forNo passphrase)://The password can be emptyEnter same Passphrase again:your identification has been savedinch/root/.ssh/id_rsa.//private KeyYour PublicKey has been savedinch/root/.ssh/id_rsa.pub.//Public KeyThe key fingerprint is: Sha256:uumjxnrh7hykokk4iqfx/Bxc3kgyvhx0glakoygodja [email protected]the key's Randomart image is:+---[RSA2048]----+|                 ||                 ||   ||  . .. O | | E..      o+ S. ||        . =*++o+. || =o.*b.oo | |        @ +.ox++. || @* o=o+ |+----[SHA256]-----+[[Email protected]~]# will now be in/root/generate the key pair in the. SSH directory [[email protected]. ssh]# ls-Latotal -drwx------2Root root4096Jul4  ,: +. Dr-xr-x---.7Root root4096Jul4  ,: - ..-RW-------1Root root0Jul3  ,: -Authorized_keys-RW-------1Root root1675Jul4  ,: +Id_rsa-rw-r--r--1Root root410Jul4  ,: +id_rsa.pub-rw-r--r--1Root root176Jul3  the: -known_hosts The second step, upload the key to the file source server ssh on the target side-copy-id [email protected]120.77.250.120[[email protected]. ssh]# SSH-copy-id [email protected]120.77.250.120/usr/bin/ssh-copy-id:info:source of key (s) to BES installed:"/root/.ssh/id_rsa.pub"The authenticity of host'120.77.250.120 (120.77.250.120)'Can't be established.ECDSA Key Fingerprint issha256:okdqrmfzve4+BVY0ACRDIKPNXE3E20BAC2G7GG0VKDQ.ECDSA Key Fingerprint isMD5: the: a4:7a: the:d B:Ten: a5:db:8d: -: BC: at: Cb:4f:a9:eb. is you sure-want toContinueConnecting (yes/no)?Yes/usr/bin/ssh-copy-id:info:attempting to loginchWith theNewKey (s), to filter outAny that is already installed/usr/bin/ssh-copy-id:info:1Key (s) remain to be installed--ifYou is prompted now it isTo install theNewKeys[email protected]120.77.250.120's Password:Number of key (s) added:1 NowTryLogging into the machine, with:"ssh ' [email protected] '"and check to make sure that is only the key (s) wanted were added. The third step, test the synchronization file in/www/wwwroot/file inside new rsync-avzp/www/wwwroot/file/[email protected]120.77.250.120:/www/wwwroot/file///No password is prompted when using a secret-free methodSynchronization Result: (success) sending incremental file Listplus-yixueqing/Plus-yixueqing/222. Phpsent -,389Bytes Received756bytes -,763.33bytes/sectotal Size is  the,583,590Speedup is 1,454.64[[email protected] file]# rsync-avzp/www/wwwroot/file/[email protected]120.77.250.120:/www/wwwroot/file/sending incremental file listsent -,342Bytes Received726bytes -,712.00bytes/sectotal Size is  the,583,590Speedup is 1,455.87Third, rsync Daemon (recommended)1To receive files on the target side, you need to configure two files to install yum install rsync-y uninstall yum remove rsync-y First, configuration file/etc/rsyncd.conf Write the module, assign the virtual user name and password and other information uid=Rootgid=RootUse chroot=Nomax Connections= $Timeout= -pid File= /var/run/Rsyncd.pidLockFILE =/var/run/rsync.Locklog File= /var/log/Rsyncd.log[faceke] Path=/www/wwwroot/file/Ignore errors Read only=No list=falseAuth Users=Anson Secrets File=/etc/Rsync.password Configuration Description: Vim/etc/Rsyncd.confuid=rsync #用户idgid=Rsyncuse chroot=no #安全性, the intranet is generally not considered, set to Nomax connections= $#最多有多少个客户端连接我timeout= -#超时时间, s PID file= /var/run/rsyncd.pid #pid文件LockFILE =/var/run/rsync.Lock#传输时会给文件加锁log File= /var/log/rsyncd.log #日志文件 [test] #模块path=/test/#客户端来同步, is to synchronize the directory ignore errors #传输过程中遇到错误, automatically ignore read only=false#可读可写list=false#不允许列表hosts Allow=10.0.0.0/ -#允许的IP段hosts deny=0.0.0.0/ +#拒绝auth Users=rsync_backup #这是个虚拟用户 (Custom) Secrets file=/etc/rsync.password #虚拟用户对应的密码文件第二, configure password file/etc/Rsync.password (write virtual user name and password) #仅写入和服务端虚拟用户对应的密码echo"anson:hao123456">/etc/rsync.password# permissions to modify the password file (required) chmod -/etc/Rsync.password Viewing the end buckle 873 Check if the 873 port monitoring is turned on lsof, i[873Kill process Pkill rsync start service rsync--Daemon2Install and configure client (file source side) client only need password file, the file only holds password: Install yum install rsync-y write only the password modification corresponding to the server-side virtual user/etc/Rsync.passwordecho"hao123456">/etc/Rsync.passwordchmod -/etc/Rsync.password to directory, start sync operation Create directory, create test file CD/www/wwwroot/mkdir filevim aaa.phpvim bbb.php ... Start Synchronizing rsync-ARVZP--delete/www/wwwroot/file/[email protected]120.77.250.120:: Faceke--password-file=/etc/Rsync.password or rsync with rsync protocol-avz rsync://[email protected]/test/www/wwwroot/file/--password-file=/etc/rsync.passwordYou can also write files and directories to be excluded, one line at a A: Vim excefile.conftest.py*. Logdir1 using-exclude- fromrsync-avz--exclude- from=excefile.conf/data/[email protected]10.0. -::/test--password-file=/etc/Rsync.password Some problems troubleshooting https://yq.aliyun.com/articles/111512?t=t1//Soft Connect/rsyncd-munged/Error solutionhttp//blog.sina.com.cn/s/blog_4da051a60102wo5v.htmlWrite timed Tasks crontab-e# will now enter the VI editing interface to let you edit the work. Notice that each job is a row. 0     A* * * Mail dmtsai-s" at". bashrc# time-sharing week| "============== command line =======================" |

represents the meaning of the minute-hour date Month Week command number range0~ - 0~ at 1~ to 1~ A 0~7on the order. The number of weeks is 0 or 7 o'clock, which means "Sunday". In addition, there are some auxiliary characters, probably the following:*(asterisk) represents the meaning of acceptance at any moment. For example, the day, month, and week in the example*, it means that the follow-up order is carried out at 12:00 on the day of the week, regardless of the month. , (comma) represents the meaning of the separation period. For example, if the work to be performed is 3:00 and 6:00, it will be:0 3,6* * *command time still has five columns, but the second column is3,6, both for 3 and 6-(minus sign) represents a period of time, for example, between 8 and 12 hours of 20 points per hour of work: - 8- A* * *command carefully see the second column becomes 8- A. Represents8,9,Ten, One, Athe meaning of all works/N (slash) that n represents a number, which is the meaning of every n unit interval, for example, every five minutes:*/5* * * *command with* with/5来, can also be written as 0- -/5, meaning the same write synchronization script can be executed manually or in the directory www/wwwroot/file/Create script Rsync.shvim rsync.sh#/bin/Shrsync-ARVZP--delete/www/wwwroot/file/[email protected]120.77.250.120:: Faceke--password-file=/etc/Rsync.password//If you have more than one server, set up multiplethe Done Configuration timer task here is timed 1 minutes to execute the script once crontab-e*/1* * * * * sh/www/wwwroot/file/rsync.sh sync file size du-sh./*ls-alh compression command TAR-ZCVF plus-yixueqing-h5.zip plus-yixueqing-h5/

Multi-server file synchronization implementation