Introduction to Rsync and Crontab and automatic backup using rsync and crontab

I./Related INTRODUCTION

1/rsync

Rsync,remotesynchronize as implies know it is a remote synchronization of the software, it synchronizes files at the same time, you can keep the original file permissions, time, soft and hard links and other additional information. Rsync uses the rsync algorithm to provide a quick way for file synchronization between a client and a remote file server, and it can be transferred via SSH, so it's very confidential and it's free software.

Rsync includes some of the following features:

Can update entire directory and tree and file system;
Selective retention of symbolic chains, hard links, file belonging, permissions, equipment, and time;
For the installation, there is no special permission requirements;
For multiple files, the internal pipeline reduces the delay of file waiting;
can use rsh, SSH or direct port as the transmission port;
Support the anonymous rsync synchronization files, is the ideal image tool;

Server mode:

This mode is based on the C/S mode, in this mode, rsync enabled a daemon in the background, the daemon is permanently running on the Rsync server to receive file transfer requests, so the client can either send the file to the daemon, or request the file to the daemon process. Rsync's server model is ideal for use in offsite central backup servers or data offsite repositories.

Enterprise case: Building a remote disaster recovery backup system

In order to ensure data security, the need to establish a remote disaster recovery system, the site data in the Daily 3 o'clock in the morning back to the remote disaster recovery server, due to a large number of daily only incremental backup, only the increase of data on the day of backup, when the site fails, you can restore the maximum amount of data by backup.

Solution: Here is assumed to have a, B two Linux systems, a system as a Web server, B system as a system of remote disaster recovery backup machine, so a system is rsync server, B system is as a system remote disaster recovery backup, need to install the Rsync software on a, b system, so that The rsync daemon is run on a system, and the data specified by the A system can be backed up periodically by the system daemon Crontab on B, thus enabling remote disaster recovery of the data.

2/crontab

Crond is a daemon that is used to periodically perform certain tasks or wait for certain events under Linux, similar to Scheduled tasks under Windows, when the operating system is installed, the Service tool is installed by default and the Crond process is started automatically. The Crond process periodically checks to see if there is a task to perform and automatically executes the task if there are tasks to perform.

The task scheduling under Linux is divided into two categories, system task scheduling and user task scheduling.

System task scheduling: The work to be performed by the system periodically, such as writing cache data to hard disk, log cleanup, etc. In the/etc directory there is a crontab file, this is the System Task Scheduler configuration file.

The/etc/crontab file includes the following lines:

[Root@localhost ~]# Cat/etc/crontab

Shell=/bin/bash

Path=/sbin:/bin:/usr/sbin:/usr/bin

Mailto= "" home=/

# Run-parts

* * * * * Root run-parts/etc/cron.hourly

7 * * * Root run-parts/etc/cron.daily

4 * * 0 root run-parts/etc/cron.weekly

4 1 * * Root run-parts/etc/cron.monthly

[Root@localhost ~]#

The first four rows are the environment variables that are used to configure the Crond task to run, the shell variable specifies which shell the system will use, this is bash, and the second line of the path variable specifies the path to the System execution command. The third line of the mailto variable specifies that Crond's task execution information will be emailed to the root user, and if the value of the mailto variable is null, the task execution information is not sent to the user, and the home variable in line fourth specifies the home directory to use when executing the command or script. The meaning of line sixth to Nineth is described in detail in the next section. There's not much to say here.

User Task scheduling: Users to perform regular work, such as user data backup, scheduled email reminders and so on. Users can use the Crontab tool to customize their own scheduled tasks. All user-defined crontab files are saved in the/var/spool/cron directory. Its file name is the same as the user name.

User Permissions File:

File:

/etc/cron.deny

Description

The users listed in this file are not allowed to use the crontab command

File:

/etc/cron.allow

Description

Users listed in this file are allowed to use the crontab command

File:

/var/spool/cron/

Description

directory where all user crontab files are stored, named by user name

What the crontab file means:

In the crontab file created by the user, each line represents a task, each field of each row represents a setting, its format is divided into six fields, the first five is the time setting segment, and the sixth paragraph is the command segment to execute, in the following format:

Minute hour day Month Week command

which

Minute: Represents minutes, which can be any integer from 0 to 59.

Hour: Represents the hour, which can be any integer from 0 to 23.

Day: Represents a date, which can be any integer from 1 to 31.

Month: Represents the month, which can be any integer from 1 to 12.

Week: Represents the day of the week, which can be any integer from 0 to 7, where 0 or 7 represents Sunday.

Command: The commands to execute can be either system commands or script files that you write yourself.

In each of these fields, you can also use the following special characters:

Asterisk (*): represents all possible values, such as the month field if it is an asterisk, the command action is executed monthly after the constraints of other fields are met.

Comma (,): You can specify a list range with a comma-separated value, for example, "1,2,5,7,8,9"

Mid-Bar (-): You can use the middle bar between integers to represent an integer range, such as "2-6" for "2,3,4,5,6" forward slash (/): You can specify the interval frequency of the time with a forward slash, for example "0-23/2" means to execute every two hours. A forward slash can be used with asterisks, such as */10, if used in the minute field, indicating that every 10 minutes is performed

two use of/rsync with Contab

1/Installation

Check if rsync is installed
Not installed to install

2/Create rsync.conf file

Server side to create the rsyncd.conf file (the server can be arbitrarily specified here, the actual execution of the server and the client can copy files, so who can do the server, anyway, when the two computers need to be open, so it can be understood as a backup between two servers, it is assumed that the server side to create conf files OK)

Generate rsyncd.conf

#vi/etc/rsyncd.conf

#[global]

UID = root

GID = root

Use chroot = no

Max connections = 10

List = yes

PID file =/var/run/rsyncd.pid

Lock file =/var/run/rsyncd.lock

Log file =/var/log/rsyncd.log

Hosts allow = 192.168.1.70//allows access to IP, which is the client IP

[Data]//Specifies the name of the publication to use when the client calls Rsync

Path =/root/test//Publish the directory that you want to back up, and this directory will be backed up to the client

Read Only = no

Ignore errors

Auth users = root//authenticated user is root

Secrets file =/etc/sery.pass//password files

#wq

#rsync –daemon--config=/etc/rsyncd.conf//Start rsync server

3/Password

Generate client and server-side password files

Client:

#vi/etc/sery_client.pass

123456

#chmod 600/etc/sery_client.pass

Service side:
#vi/etc/sery.pass

root:123456# note. Server side has a user name, after the client password, permissions are 600, and the previous conf file is also

#chmod 600/etc/sery.pass

4/Start rsync

Server-side Startup
# rsync--daemon--config=/etc/rsyncd.conf

5/View listening status

Also server-side
#lsof –i:873

6/write rsync into the server self-boot

or the server-side
# echo "/usr/bin/rsync--daemon--config=/etc/rsyncd.conf" >>/etc/rc.local

7/Test Sync

This is the client.
#rsync –avzp–-delete root@192.168.1.64::d ata/root/test–-password-file=/etc/

Sery_client.pass

1.–a use archive mode to maintain the original file permissions

2.–v Display on the screen

3.–z compressing data while transmitting

4.–p Transfer Progress

5.--delete means that if the server side deletes the file, then the client also deletes the file accordingly, maintaining true consistency

6. Data is the name of the previous publication,/root/test client local storage path

7.–password-file Specifying a password file

Appear:
[Root@stserzzptora-bak data]# rsync-avzp--delete root@192.168.1.64::d ata/root/test/data--password-file=/etc/sery_ Client.pass

Receiving file List ...

7 Files to consider

./Files ...

6.txt

0 100% 0.00kb/s 397113:04:51 (1, 100.0% of 7)



Sent 124 bytes received bytes 728.00 bytes/sec

Total size is one speedup is 0.03

Indicates that the client and the server are connected properly.

8/crontab

Write into crontab on the client

#vi/root/rsyncd.sh

#!/bin/bash

Rsync–avzp–-delete root@192.168.1.64::d ata/root/test/data–-password-file

=/etc/sery_client.pass

#wq

Check to see if the synchronization is normal.

9/Other

(1) is the client and server-side connection can also use SSH link without password method

Automatic Ssh/scp method = =
A is a local host (that is, a machine used to control other hosts); 192.168.4.194
b is a remote host (that is, the controlled machine server), 192.168.4.193

Systems A and B are all Linux.

Run the command on a:
# ssh-keygen-t RSA (three consecutive returns, i.e. locally generated public and private keys, no password set)
# SSH root@192.168.4.193 "mkdir. SSH" (password required)
# SCP ~/.ssh/id_rsa.pub root@192.168.4.193:.ssh/id_rsa.pub (need to enter password)

command on B:
# Touch/root/.ssh/authorized_keys (if this file already exists, skip this one)
# cat/root/.ssh/id_rsa.pub >>/root/.ssh/authorized_keys (append id_rsa.pub content to Authorized_keys)

Back to a machine:
# SSH root@192.168.4.193 (no password required, login successful)

A simpler approach

Open terminal execution Ssh-keygen, which creates Id_rsa, id_rsa.pub two files in the ~/.ssh/directory, respectively, for your public and private keys.

Copy the public key to the server's ~/.ssh/authorized_keys file. There are several ways to copy a public key through the SCP to the server and then append it to the ~/.ssh/authorized_keys file, which is more cumbersome. Scp-p ~/.ssh/id_rsa.pub user@host:~/. With the Ssh-copyid program, Ssh-copyid User@host is available, but this method does not support changing the port number (I did not find it). The program Ubuntu system comes with no installation, in fact the program is a script. Available with Cat ~/.ssh/id_rsa.pub | Ssh-p user@host ' cat >> ~/.ssh/authorized_keys ', this is one of my more common methods, because you can change the port number.

three/commands about rsync

The command format for rsync can be in the following six ways:
rsync [OPTION] ... SRC DEST
rsync [OPTION] ... SRC [User@]host:dest
rsync [OPTION] ... [user@] HOST:SRC DEST
rsync [OPTION] ... [user@] HOST::SRC DEST
rsync [OPTION] ... SRC [user@]host::D EST
rsync [OPTION] ... rsync://[user@]host[:P ort]/src [DEST]
There are six different modes of operation for Rsync, which corresponds to the above six command formats:
1) Copy the local file. This mode of operation is initiated when both the SRC and des path information do not contain a single colon ":" delimiter. such as: Rsync-a/data/backup
2) Use a remote shell program (such as rsh, SSH) to copy the contents of the local machine to the remote machine. This mode is started when the DST path address contains a single colon ":" delimiter. such as: Rsync-avz *.c foo:src
3) Use a remote shell program (such as rsh, SSH) to copy the contents of the remote machine to the local machine. This mode is started when the SRC address path contains a single colon ":" delimiter. such as: Rsync-avz Foo:src/bar/data
4) Copy files from the remote rsync server to the local machine. This mode is started when the SRC path information contains the "::" delimiter. such as: Rsync-av Root@172.16.78.192::www/databack
5) Copy files from the local machine to the remote rsync server. This mode is started when the DST path information contains the "::" delimiter. such as: Rsync-av/databack root@172.16.78.192::www
6) The list of files for the remote machine is listed. This is similar to the rsync transfer, but only if the local machine information is omitted from the command. such as: Rsync-v rsync://172.16.78.192/www

The specific explanations for the rsync parameters are as follows:
-V,--verbose verbose mode output
-Q,--quiet thin output mode
-C,--checksum turn on the check switch to force verification of file transfers
-A,--archive archive mode, which means to transfer files recursively and keep all file attributes equal to-rlptgod
-R,--recursive subdirectories in recursive mode
-R,--relative using relative path information
-B,--backup creates a backup, that is, the old file is renamed to ~filename when the same file name exists for the purpose. You can use the--suffix option to specify a different backup file prefix.
--backup-dir store backup files (such as ~filename) in the directory.
-suffix=suffix Defining backup File prefixes
-U,--update only updates, which is to skip all the files that already exist in DST, and the file time is later than the file to be backed up. (Does not overwrite the updated file)
-L,--links reserved Soft link
-L,--copy-links to handle soft links like regular files
--copy-unsafe-links only copies links to links outside the SRC Path directory tree
--safe-links ignoring links to the SRC Path directory tree
-H,--hard-links reserved Hard link
-P,--perms maintain file permissions
-O,--owner keep file owner information
-G,--group keep file group information
-D,--devices keep device file information
-T,--times keep file time information
-S,--sparse special processing of sparse files to save DST space
-N,--dry-run reality which files will be transmitted
-W,--whole-file copy files without incremental detection
-X,--one-file-system do not cross file system boundaries
-B, the block size used by the--block-size=size test algorithm, is 700 bytes by default
-E,--rsh=command specifies data synchronization using RSH and SSH
--rsync-path=path Specify the path information for the rsync command on the remote server
-C,--cvs-exclude uses the same method as CVs to automatically ignore files to exclude files that you do not want to transfer
--existing only updates those files that already exist in DST without backing up those newly created files
--delete Delete those files that are not in the DST SRC
--delete-excluded also deletes those files that are excluded by the option specified by the Receive side
--delete-after transfer ends after removal
--ignore-errors Timely IO errors are also deleted
--max-delete=num deleting NUM files up to a maximum
--partial retains files that are not fully transmitted for any reason, to expedite subsequent transmissions
--force forcibly delete a directory, even if it is not empty
--numeric-ids does not match the user and group ID of a number to a user name and group name
--timeout=time IP time-out, in seconds
-I,--ignore-times do not skip files that have the same time and length
--size-only when deciding whether to back up a file, just look at the file size regardless of file time
--modify-window=num determines whether the file is time-stamped with the time Stamp window, which defaults to 0
-t--temp-dir=dir creating temporary files in Dir
--compare-dest=dir also compares the files in DIR to determine if a backup is required
-p equivalent to--partial
--progress Show Backup process
-Z,--compress compress the backed-up files during transmission
--exclude=pattern specify to exclude file modes that do not need to be transferred
--include=pattern specifies file modes that need to be transferred without exclusion
--exclude-from=file exclude files in the specified schema in file
--include-from=file does not exclude files that specify pattern matching
--version Print version Information
--address binding to a specific address
--config=file specify a different configuration file, do not use the default rsyncd.conf file
--port=port Specify a different rsync service port
--blocking-io using blocking IO for remote shells
-stats gives the transfer status of some files
--progress in the transmission of the real-time transmission process
--log-format=format specifying the log file format
--password-file=file get the password from file
--bwlimit=kbps limit I/O bandwidth, Kbytes per second
-H,--help display Help information

four/about the use of crontab

Basic format:
* * * * * command
Time-sharing Weekly command
The 1th column represents minutes 1~59 per minute with * or */1
The 2nd column represents the hour 1~23 (0 means 0 points)
The 3rd column represents the date 1~31
The 4th column represents the month 1~12
5th Column Identification Number Week 0~6 (0 = Sunday)
6th List of commands to run
Some examples of crontab files:
* * * * */usr/local/etc/rc.d/lighttpd restart
The above example shows that 21:30 restarts Apache per night.
4 1,10,22 * */USR/LOCAL/ETC/RC.D/LIGHTTPD restart
The above example shows that 4:45 restarts Apache on the 1, 10, and 22nd of the month.
1 * * 6,0/USR/LOCAL/ETC/RC.D/LIGHTTPD restart
The above example shows that 1:10 restarts Apache every Saturday and Sunday.
0,30 18-23 * * */usr/local/etc/rc.d/lighttpd restart
The above example shows that Apache restarts every 30 minutes from 18:00 to 23:00 every day.
0 * * 6/USR/LOCAL/ETC/RC.D/LIGHTTPD restart
The above example shows the restart of Apache every Saturday at 11:00am.
* */1 * * * */usr/local/etc/rc.d/lighttpd restart
Restart Apache every hour
* 23-7/1 * * * */usr/local/etc/rc.d/lighttpd restart
From 11 o'clock to 7 in the morning, restart Apache every hour.
0 4 * mon-wed/usr/local/etc/rc.d/lighttpd restart
4th per month with 11-point restart from Monday to Wednesday Apache
0 4 1 Jan */usr/local/etc/rc.d/lighttpd restart
4-point restart of Apache on January 1
Name: crontab
Usage rights: All users
How to use:
crontab file [-u user]-replaces the current crontab with the specified files.
Crontab-[-u user]-replaces the current crontab with standard input.
crontab-1[user]-lists the user's current crontab.
crontab-e[user]-Edit user's current crontab.
crontab-d[user]-Delete the user's current crontab.
Crontab-c dir-Specifies the directory for crontab.
Format of the crontab file: M H d M D cmd.
M: Minutes (0-59).
H: Hours (0-23).
D: Day (1-31).
M: Month (1-12).
D: Day of the Week (0~6,0 is Sunday).
CMD to run the program, the program is fed into sh execution, this shell only user,home,shell these three environment variables
Description
Crontab is used to allow a user to execute a program at a fixed time or at a fixed interval, in other words, a user-like schedule. -u user refers to setting the specified
User's time table, which presupposes that you must have permission (for example, root) to specify another's time schedule. If you do not use the-u user, it means to set the
Set your own schedule.
Parameters:
CRONTAB-E: Do a text editor to set the time table, the default text editor is VI, if you want to use another text editor, please set the VISUAL environment variables first
To specify the use of that text editor (for example, Setenv VISUAL Joe)
Crontab-r: Delete the current schedule table
Crontab-l: List the current schedule
crontab file [-u user]-replaces the current crontab with the specified files.
The format of the schedule table is as follows:
F1 F2 F3 f4 f5 program
Where F1 is expressed in minutes, F2 represents hours, F3 represents the day of the month, F4 represents the month, and F5 represents the day of the one week. program says
The program of the line.
When F1 is * indicates that every minute the program,f2 is executed, the program is executed every hour, and the remainder of the analogy
When the F1 is a-B, it is executed from the time of the minute to the minute of the hour, and the F2 is a-B indicating that it is executed from the first to the first, and the remainder of the analogy
When F1 is */n, it is performed once every n minutes, and F2 is */n for every n-hour interval, with the remainder of the other analogy
When F1 is a, B, C,... A, B, C,... Minutes to execute, F2 for a, B, C,... The time indicated that the first, B, c ... Hours to execute, and the rest of the analogy
The user can also store all the settings in the file file, using the crontab file to set the time schedule.
Example:
#每天早上7点执行一次/bin/ls:
0 7 * * */BIN/LS
In December, the/usr/bin/backup is performed every 3 hours in the morning from 6 to 12.
0 6-12/3 */usr/bin/backup
Monday to Friday every 5:00 send a letter to Alex@domain.name:
0 * * 1-5 mail-s "HI" Alex@domain.name </tmp/maildata
Every day of the month 0:20 midnight, 2:20, 4:20 .... Perform echo "haha"
0-23/2 * * * echo "haha"
Attention:
When the program is executed at the time you specify, the system will send you a letter showing what the program is doing, and if you do not wish to receive such a letter, please leave a blank in each line
After adding >/dev/null 2>&1 can be
Example 2:
#每天早上6点10分
6 * * * Date
#每两个小时
0 */2 * * * Date
#晚上11点到早上8点之间每两个小时, 8 in the morning.
0 23-7/2,8 * * * Date
#每个月的4号和每个礼拜的礼拜一到礼拜三的早上11点
0 4 * mon-wed date
#1月份日早上4点
0 4 1 Jan * Date
Example
$crontab-L lists the user's current crontab.

Read more about this link http://www.cnblogs.com/peida/archive/2013/01/08/2850483.html

There's sketch in it.