In Windows 7, how does one use multithreading to accelerate File Replication?

Source: Internet
Author: User
Tags robocopy

Why cannot I speed up file copy/transfer in Windows 7 in Windows 7? AvailableRobocopyImplement multi-threaded file replication.
Many people who often perform file management and copy/paste operations may recall that Vista was a very slow copy speed and was not satisfied with the built-in copy function of windows, because it is too fast. Therefore, we used software such as fastcopy and teracopy to accelerate replication. Windows 7 then fixed this problem, but the effect seemed unsatisfactory, but do you know that Windows 7 has a built-in quick copy function?
If you want to copy a directory containing multiple large-capacity files, you need to use the robocopy command, which is a command line Directory Copy command, since Windows NT 4.0, it has become part of the Windows resource toolkit, and is then built into Windows Vista, Windows 7, and Windows Server 2008 as a standard feature. This tool not only copies fast, resumable data transfer is also supported, and you can even copy data at specified time. This feature in Windows 7 has also been upgraded-multithreading is supported, which means the replication speed can be greatly improved. In addition, this function allows you to create two copies of an image with a complete file structure, without copying unnecessary duplicate files. It also allows you to retain all relevant file information, including date and timestamp, security access control list (ACL), and so on. Copying N-plus small file directories can increase the speed by an order of magnitude. Thousands of files are copied in 3 seconds, and the commonly used copy and paste method takes about 45 seconds.
For exampleF :\All files inEDisk\ CDDirectory, you can run the following command:
Robocopy/s f: \ E: \ CD
Even if the disc is suddenly removed at this time, robocopy will wait until the disc is re-inserted into the optical drive and the copy will automatically start.
Robocopy also supports local area network copy, and even the directory structure of the specified directory on the local area network server can be moved to the local machine. robocopy supports regular operations, so that the replication operation can be performed at night, to avoid LAN congestion and inconvenience to others.
Next, let's take a look at the usage of this command, as shown in:

For example, robocopy dir_from dir_to/e/MT: 50/log: Copy. Log
/E indicates that the directory contains sub-directories./MT: 50 indicates that 50 threads are used (8 threads by default, up to 128 threads can be set), And/log indicates that the output is redirected. For more options, use robocopy /? View.
Command syntax: robocopy source destination...] [Options]
Source: Source directory (drive: \ path or \ Server \ share \ path)
Target: target directory (drive: \ path or \ Server \ share \ path)
File: the file to be copied (name/wildcard: "*. *" by default "*.*")
To test the speed of copying files, the following tests are performed:

We can see that, in the case of 120 threads, a single file with a size of 878.16 MB has been copied, starting from 14:56:31 and ending at 14:57:06, sharing 35 seconds.
To test how many threads can be selected to copy files more quickly, use powershell scripts for batch processing to test the time needed for copying to different threads one by one, and plot the points on the coordinates. The powershell script for the test is as follows:
For ($ I = 128; $ I-lt 129; $ I ++ ){
Robocopy/s c: \ cd c: \ test \ CD $ I/MT: $ I/log: log/copy $ I. Log
Remove-item c: \ test \ CD $ I \*.*
"$ I,"> log/answ.txt
(Get-content c: \ log \ Copy $ I. Log-totalcount 6) [-1]> log/answ.txt
","> Log/answ.txt
(Get-content c: \ log \ Copy $ I. Log-totalcount 10000) [-1]> log/answ.txt}
The content output by the script is formatted in a regular way, imported to excel in CSV format, and the line chart is shown in"Series 1"Line:

There are a total of 324 files in the target directory to be copied. The size of a single file is between 1 MB and 2 MB, and the total size of the file is 549 MB, the copy time is longer than before, because there are many files, it takes a long time to copy them one by one, and there will be errors after many experiments, after normalization, the yellow "linear (Series 1)" line shows that the copy speed of the same file group varies with different threads. As the number of threads increases, the copy time is linearly reduced.
In fact, there are many powerful ways to use robocopy. You can use robocopy /? Command to query, or view the detailed options of the robocopy command from the attachment at the end of this article, the reader will find that this command has more powerful functional options.
Appendix: detailed options of the robocopy command
Copy Option
/S: Copy sub-directories, but do not copy empty sub-directories.
/E: Copy sub-directories, including empty sub-directories.
/Lev: N: only copies the first N layers of the source directory tree.
/Z: copy the file in restart mode.
/B: Copy files in backup mode.
/ZB: Use the restart mode. If access is denied, use the backup mode.
/Efsraw: Copies all encrypted files in efs raw mode.
/Copy: Copy Tag: content of the file to be copied (/copy: DAT by default ). (Copy Tag: D = data, a = attribute, t = timestamp ). (S = Security = ntfs acl, O = Owner information, u = audit information ).
/Dcopy: T: Specifies the copy directory timestamp.
/Sec: Copies secure files (equivalent to/copy: DATS ).
/Copyall: Copy all file information (equivalent to/copy: datsou ).
/Nocopy: do not copy any file information (used with/purge ).
/Secfix: fixes the security of all files, even the files that are skipped.
/Timfix: the time when all files are repaired, even if they are skipped.
/Purge: Delete the target file/directory that no longer exists in the source.
/MIR: Image directory tree (equivalent to/E and/purge ).
/MoV: Move the file (copy and delete it from the source ).
/Move: Move files and directories (copy and delete from the source ).
/A +: [rashcnet]: add the specified attribute to the copy file.
/A-: [rashcnet]: deletes a specified attribute from the copy file.
/Create: only create a directory tree and a file with zero length.
/Fat: only use the 8.3 FAT file name to create the target file.
/256: Disable the support for ultra-long paths (> 256 characters.
/MON: N: Monitoring source; run again when more than N changes are found.
/MOT: M: Monitoring source; If changed, run again within M minutes.
/RH: hhmm-hhmm: running hours-the time when a new copy can be started.
/PF: Check the running hours based on each file (rather than each step.
/IPG: N: The spacing (MS) between packages to release the bandwidth on low-speed lines.
/SL: Copies symbolic links against the target.
/MT [: N]: Use n threads for multi-thread replication (default value: 8 ). N must be at least 1, but not greater than 128. This option is not compatible with/IPG and/efsraw options. Use the/log option to redirect the output for optimal performance.
File selection options
/A: only copy files with an archive property set.
/M: only copy files with archive attributes and reset the archive attributes.
/IA: [rashcneto]: contains only files with any given property set.
/XA: [rashcneto]: exclude files with any given property set.
/XF file [file]...: exclude files that match the given name/path/wildcard.
/XD directory [Directory]...: Exclude directories that match the given name/path.
/XC: exclude changed files.
/XN: exclude newer files.
/XO: exclude older files.
/XX: exclude unnecessary files and directories.
/XL: exclude isolated files and directories.
/Is: contains the same file.
/It: contains the adjusted file.
/MAX: N: Maximum File Size-exclude files larger than n Bytes.
/Min: N: Minimum File Size-exclude files smaller than n Bytes.
/Maxage: N: Specifies the longest time when a file exists.-exclude files earlier than N days/date.
/Minage: N: specifies the minimum time for the file to exist-exclude files later than N days/date.
/Maxglad: N: Maximum last access date-exclude files not used since N.
/Minglad: N: Minimum last access date-exclude files used since N. (If n <1900 then n = n days, else n = yyyymmdd date ).
/XJ: exclude joining points. (This is usually included by default ).
/FFT: Assume the FAT file time (2 seconds granularity ).
/DST: Make up for the DST time difference of one hour.
/Xjd: exclude directory joining points.
/Xjf: exclude file joining points.
Retry options
/R: N: Number of Retries for failed copies: The default value is 1 million.
/W: N: the waiting time between two retries. The default value is 30 seconds.
/REG: Save/R: N and/W: N in the registry as the default setting.
/TBD: Wait for the shared name to be defined (retry error 67 ).
Logging options
/L: only list-do not copy, add timestamp, or delete any files.
/X: report all unnecessary files, not just selected files.
/V: generates detailed output and displays the skipped files.
/TS: Timestamp of the source file included in the output.
/FP: the complete path name of the file in the output.
/Bytes: print the size in bytes.
/Ns: no size-the file size is not recorded.
/NC: no category-the file category is not recorded.
/Nfl: No file list-No file name record.
/NDL: No directory list-No directory name is recorded.
/NP: No progress-the percentage of copies is not displayed.
/ETA: displays the expected time to arrive at the copied file.
/Log: file: outputs the status to the log file (overwrite existing logs ).
/Log +: file: outputs the status to the log file (attached to the existing log ).
/Unilog: file: outputs the status to the log file in Unicode mode (overwrite existing logs ).
/Unilog +: file: outputs the status to the log file in Unicode mode (appended to the existing log ).
/Tee: outputs to the console window and log files.
/NJS: No Job Summary.
/Unicode: output status in Unicode mode.
Job options
/Job: Job name: extract parameters from the named job file.
/Save: Job name: Save parameters to the named job file.
/Quit: process the command line and exit (view parameters ).
/Nosd: The Source directory is not specified.
/Nodd: The target directory is not specified.
/If: contains the following files.

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.