Web site automation backup of Shell programming application

Source: Internet
Author: User
Tags comparison table file copy inotify rsync

Rsync is a data mirroring Backup tool--remote Sync under Unix-like systems. A fast incremental backup tool remote sync that supports local replication or synchronizes with other SSH and rsync hosts. Compared with the traditional methods of CP, SCP and tar backup, rsync has the advantages of high security, fast backup, support of incremental backup, and so on, rsync can solve the requirement of low-real-time data backup, such as regular backup file server data to remote server, regular data mirroring for local disk, etc.

Environment
CentOS 5.8 (192.168.23.130) (service)
CentOS 5.8 (192.168.23.131) (client)
Software
Rsync-3.1.1.tar.gz (http://rsync.samba.org/)
Installation steps
1. Pre-preparation conditions
A. Turn off SELinux, firewalls (personal habits whatever the case, you must close your own play)
B.yum install-y make GCC
2. Install rsync (service, client)
A.TAR-ZXF rsync-3.1.1.tar.gz
B.CD rsync-3.1.1
C../configure--prefix=/usr/local/rsync
D.make
E.make Install
3. Configuring rsync
a.vim/etc/rsyncd.conf (Service)
UID = root
GID = root
Use chroot = no
[Apps]
Auth users = root
Secrets file =/etc/rsyncd.passwd
Munge symlinks = No
Read Only = no
Path=/data/test
B.VIM/ETC/RSYNCD.PASSWD (Service)
root:111111
C.VIM/ETC/RSYNC.PASSWD (client)
111111
4. Start Rsync (service)
/usr/local/rsync/bin/rsync--daemon--config=/etc/rsyncd.conf
5. Synchronizing files (client)
/USR/LOCAL/RSYNC/BIN/RSYNC-VZRTOPG--progress--delete rsync://[email protected]/apps/tmp/tmp/--password-file=/etc /rsync.pas (service->client)
/USR/LOCAL/RSYNC/BIN/RSYNC-VZRTOPG--progress--delete/tmp/tmp/rsync://[email Protected]/apps--password-file=/etc /rsync.pas (Client->service)
6. Verification
To the appropriate directory to see if a synchronized file exists
After the manual run, for the more lazy people certainly not suitable, the following introduction Inotify-tools tools, can realize real-time automatic synchronization. Inotify is a Linux feature that monitors file system operations such as read, write, and create. The Inotify is responsive, very simple to use, and much more efficient than the busy polling of cron tasks. Learn how to integrate INotify into your application and discover a set of command-line tools that you can use to further automate system governance.
7. Installing Inotify-tools
A.wget http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
B.TAR-ZXF inotify-tools-3.14.tar.gz
C.CD inotify-tools-3.14
D../configure--prefix=/usr/local/inotify
E.make
F.make Install
8. Writing shell scripts (rsync.sh)
#!/bin/sh #local dir dstdir= "/tmp/tmp/" #sync user rsyncuser= "root" #sync pas Sword rsyncpassword= "/etc/rsync.pas" #remote IP remoteip= "192.168.23.131" #remote module module= "Apps" #   Sync remote server module to local dir for IP in $remoteip do/usr/local/rsync/bin/rsync-vzrtopg--progress --delete [email protected] $ip:: $module $dstdir--password-file= $rsyncpassword done #monitor local dir, the  n rsync remote server Module/usr/local/inotify/bin/inotifywait-mrq--timefmt '%d/%m/%y%h:%m '--format '%T%w%f%e '-E  Close_write,modify,delete,create,attrib,move $dstdir | While read file does for IP in $remoteip do/usr/local/rsync/bin/rsync-vzrtopg--progre SS--delete $dstdir [email protected] $ip:: $module--password-file= $rsyncpassword echo "${file} WA S rsynced ">>/tmp/rsync.log 2>&1 done 


9.chmod 755 rsync.sh
10. Run the shell script and go to the appropriate directory operation to see if the operation was successful
11. Set up boot from boot

echo "Sh/usr/local/inotify/rsync.sh &" >>/etc/rc.d/rc.local


Divergence: If there are more than one Web server on the line, we can upload files in bulk via Rsync+inotify. The specific approach is to determine a server as a content distributor, all the online servers are rsync+inotify to monitor the content distribution machine, once the update is in real-time synchronization to the line, this is a small automated operation.


Command parameter description
1.rsyncd.conf detailed
Global parameters
UID = root//user running the rsync daemon
GID = root//group running the rsync daemon
Use chroot = no//no chroot
Max connections = 4//maximum number of connections is 4
Strict modes =yes//Whether to check the password file permissions
Port = 873//default port 873

Module parameters
[Backup]//Here is the module name of the authentication, on the client side need to specify
Path =/home/backup///The directory that needs to be mirrored, indispensable!
Comment = This is a test//comment information for this module
Ignore errors//can ignore some unrelated IO errors
Read Only = yes//read-only
List = no//Do not allow column files
Auth users = hening//authenticated user name, if no this line indicates is anonymous, this user is not system-independent
Secrets file =/etc/rsync.pas//password and user name comparison table, password files generated by themselves
Hosts allow = 192.168.1.1,10.10.10.10//allows host
Hosts deny = 0.0.0.0/0//Disallow Host
#transfer logging = yes
Note: The following green files are files that are automatically generated after the Rsync service is installed
Location of pid file =/var/run/rsyncd.pid//pid files
Lock file =/var/run/rsync.lock//Lock files are stored in the location
Log file =/var/log/rsyncd.log//Where the log files are stored
2.rsync Command Parameters
-V,--verbose verbose mode output
-Q,--quiet thin output mode
-C,--checksum turn on the check switch to force verification of file transfers
-A,--archive archive mode, which means to transfer files recursively and keep all file attributes equal to-rlptgod
-R,--recursive subdirectories in recursive mode
-R,--relative using relative path information
-B,--backup creates a backup, that is, the old file is renamed to ~filename when the same file name exists for the purpose. You can use the--suffix option to specify a different backup file prefix.
--backup-dir store backup files (such as ~filename) in the directory.
-suffix=suffix Defining backup File prefixes
-U,--update only updates, which is to skip all the files that already exist in DST, and the file time is later than the file to be backed up. (Does not overwrite the updated file)
-L,--links reserved Soft link
-L,--copy-links to handle soft links like regular files
--copy-unsafe-links only copies links to links outside the SRC Path directory tree
--safe-links ignoring links to the SRC Path directory tree
-H,--hard-links reserved Hard link
-P,--perms maintain file permissions
-O,--owner keep file owner information
-G,--group keep file group information
-D,--devices keep device file information
-T,--times keep file time information
-S,--sparse special processing of sparse files to save DST space
-N,--dry-run reality which files will be transmitted
-W,--whole-file copy files without incremental detection
-X,--one-file-system do not cross file system boundaries
-B, the block size used by the--block-size=size test algorithm, is 700 bytes by default
-E,--rsh=command specifies data synchronization using RSH and SSH
--rsync-path=path Specify the path information for the rsync command on the remote server
-C,--cvs-exclude uses the same method as CVs to automatically ignore files to exclude files that you do not want to transfer
--existing only updates those files that already exist in DST without backing up those newly created files
--delete Delete those files that are not in the DST SRC
--delete-excluded also deletes those files that are excluded by the option specified by the Receive side
--delete-after transfer ends after removal
--ignore-errors Timely IO errors are also deleted
--max-delete=num deleting NUM files up to a maximum
--partial retains files that are not fully transmitted for any reason, to expedite subsequent transmissions
--force forcibly delete a directory, even if it is not empty
--numeric-ids does not match the user and group ID of a number to a user name and group name
--timeout=time IP time-out, in seconds
-I,--ignore-times do not skip files that have the same time and length
--size-only when deciding whether to back up a file, just look at the file size regardless of file time
--modify-window=num determines whether the file is time-stamped with the time Stamp window, which defaults to 0
-t--temp-dir=dir creating temporary files in Dir
--compare-dest=dir also compares the files in DIR to determine if a backup is required
-p equivalent to--partial
--progress Show Backup process
-Z,--compress compress the backed-up files during transmission
--exclude=pattern specify to exclude file modes that do not need to be transferred
--include=pattern specifies file modes that need to be transferred without exclusion
--exclude-from=file exclude files in the specified schema in file
--include-from=file does not exclude files that specify pattern matching
--version Print version Information
--address binding to a specific address
--config=file specify a different configuration file, do not use the default rsyncd.conf file
--port=port Specify a different rsync service port
--blocking-io using blocking IO for remote shells
-stats gives the transfer status of some files
--progress in the transmission of the real-time transmission process
--log-format=format specifying the log file format
--password-file=file getting password permissions from file 600
--bwlimit=kbps limit I/O bandwidth, Kbytes per second
inotifywaitParameters
-M is to keep listening
-R is a recursive view directory
-Q is the print out event
-e Create,move,delete,modify,attrib refers to the "Listen for create mobile Delete Write permission" event

Web site automation backup of Shell programming application

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.