Differential file comparison and extraction in Linux folders-the use of rsync

Differential file comparison and extraction in Linux folders-the use of rsync
Requirement

Recently, the team is developing a version comparison tool. The requirement is to compare the A1 folder with A2, and output the incremental part of A2 to the update folder to generate the incremental update package/differential package.

Scheme Research

The first response to this function is to traverse two folders, compare a single file to see if it exists in MD5 comparison, and output the results, but this is definitely the lowest practice, and the time complexity is O (n²), which is basically a direct denial.

Later, I thoroughly studied the diff + patch solution commonly used in Kernel patches, that is

diff -urNa dir1 dir2

After a differential file is generated, use the patch command to copy the file or modify the file content. It is obvious that if you want to implement the requirement, you still need to manually parse the diff result. It is also complicated.

Suddenly .. Or the flash effect of the masaka expression in the cartoon, similar to that in the cartoon...

It can analyze folder differences, support multiple comparison modes checksum, mod-time, and size, and filter out binary files and intermediate files that svn does not need to submit, it is often used for incremental synchronization on the server, not rsync... (The rsync font size should be increased by 10 times, but the markdown cannot be changed)

The only problem to be solved now is that rsync synchronizes data from folder A to folder B. Can I redirect the synchronized file to folder C without changing the content of folder B?

After reading the rsync man ~ The -- dry-run parameter is used to demonstrate trial .. That is to say, if you use dry-run, you can only view it and it will not change...
Start coding now...

Step 1

A1.1 is a new folder, A1.0 is an old folder, and out is an incremental file that stores the directory structure from A1.0 to A1.1.

rsync --dry-run -rcnC --out-format="%n"  A1.1/  A1.0/

Note that the-C parameter is used to filter files that do not need to be synchronized according to the cvs ignore rule. In CVS, binary files are not submitted by default. Therefore, if binary files need to be extracted, then do not add C.
After the command is executed, a file list of the A1 directory is obtained:
A1.1/system/app/
A1.1/system/app/A.apk
A1.1/system/app/B .apk
A1.1/system/app/C.apk
That is, the differential file. Because the folder is displayed by default, You need to filter out the display of folders ending "/".

rsync --dry-run -rcnC --out-format="%n"  A1.1/  A1.0/ |grep -v "/$"

In this way, a list of pure incremental files is obtained.

Step 2

With the file list, there are a lot of copying operations. You can continue to use rsync to synchronize data to the new out directory to prevent file errors after multiple executions.
Complete code

rsync --dry-run -rcnC --out-format="%n"  A1.1/  A1.0/ |grep -v "/$"|xargs -I{} rsync -R A1/./{} out/

Done!

Summary

In fact, we have many commonly used tools and powerful functions. In fact, as a programmer, "impetuous" is a boundary that separates experts from cainiao. In fact, the master doesn't know how many languages, how many modes are understood, and how many open source code tools are added to the favorites...
Instead, we need to take every step in a down-to-earth manner.

RSync for file backup Synchronization

Monitor host files and directories using inotifywait

Using inotify + rsync for Linux File batch update

Inotify-tools + rsync real-time file synchronization installation and configuration

Complete rsync synchronization Configuration

Remote synchronization of Rsync in CentOS 6.5

Rsync details: click here
Rsync: click here

This article permanently updates the link address: