Hyper-V Server data deduplication technology

Hyper-V Server data deduplication technology

Swaiiow heard that the new technology in Windows Server 2012 is called Deduplication, which is said to save disk space significantly, and let's look at what deduplication is:

Data deduplication refers to finding and deleting duplicates in the data without affecting their fidelity or integrity. The goal is to change (32-128 KB) small chunks by splitting the files into sizes, identify duplicate chunks, and then keep a copy of each chunk to store more data in a smaller space. A redundant copy of a chunk is replaced by a reference to a single copy. The chunks are compressed and then organized into the System Volume Information folder as a special container file:

650) this.width=650; "height=" 347 "title=" clip_image002 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image002 " Src= "http://s3.51cto.com/wyfs02/M01/77/F1/wKiom1ZyBhDguHz7AADtXJ8ZlU0747.jpg" border= "0"/>

Windows Server 2012/R2 begins to integrate data deduplication technology, and by using the built-in deduplication capabilities of Windows Server 2012/R2, businesses can dramatically improve the efficiency of their storage space usage. For most enterprise IT departments, storage efficiency is a real big problem, because the speed at which storage costs fall is far from offsetting the growth in data volumes. To reduce the need to increase storage space, it is necessary to improve the efficiency of data storage, whether the data is stored in the data storage, or through the Wide Area Network (WAN) to move, is a big problem. To respond to this growth, the Enterprise IT department consolidates the file servers. At the same time, storage expansion and optimization are also a primary goal of their storage consolidation platform.

To address the growth of enterprise data storage, administrators want to consolidate multiple servers and set capacity scaling and data optimization as key goals. The data deduplication feature provides a viable way to achieve these goals, including:

1. Capacity optimization: Data deduplication in Windows 8.1&server 2012 stores more data in less physical space. It can achieve greater storage efficiency than single-instance storage (SIS) or NTFS compression capabilities. The Deduplication feature uses sub-file variable-size chunks and compression, and the general file server has a total optimization rate of 2:1, while the optimization rate for virtual data is up to 20:1.

2. Scalability and performance: in Windows 8.1& Server 2012, the deduplication feature is highly scalable, effectively leveraging resources, and without interference. It can process approximately MB of data per second and can run on multiple volumes at the same time without impacting other workloads on the server. Maintain a low impact on server workloads by limiting the consumption of CPU and memory resources. If the server is too busy, the data deduplication feature may stop completely. In addition, administrators are more flexible: You can run data deduplication at any time, set a run schedule for data deduplication, and establish a selection strategy.

3. Reliability and data integrity: Data integrity is maintained when you apply data deduplication. Windows 8.1&server 2012 uses checksum, consistency, and authentication to ensure data integrity. Also, for all metadata and most commonly referenced data, deduplication remains redundant, ensuring that data is recoverable when data is corrupted.

4. Improve bandwidth efficiency with BranchCache: The same optimization techniques can be applied to data transferred over the WAN to the branch office through integration with BranchCache. The result is shorter file download times and lower bandwidth usage.

650) this.width=650; "height=" 227 "title=" clip_image003 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image003 " Src= "Http://s3.51cto.com/wyfs02/M02/77/F1/wKiom1ZyBhKiDKTtAADYc0Jzpnk188.png" border= "0"/>

5. Use familiar tools for optimal management: Windows 8.1&server 2012 has the optimizations built into Server Manager and Windows PowerShell. The default setting enables immediate savings, and administrators can fine-tune the settings for more savings. Users can easily use Windows PowerShell cmdlets to start an optimization job or plan to run in the future. You can also use the Unattend.xml file (which invokes Windows PowerShell scripts and is used with Sysprep to deploy deduplication when the system first starts) to install the Deduplication feature and enable Deduplication on the selected volume.

After you enable deduplication for a volume and optimize the data, the volume contains the following:

1. Files that are not optimized: for example, files that are not optimized can include files that do not meet the selected file Retention policy settings, System State files, alternate data streams, encrypted files, files with extended attributes, files less than KB, other re-profiling point files, or files that are being used by other applications.

2. Optimized file: A file stored as a re-analysis point that contains pointers to maps for each chunk in the chunk store that are required to restore the requested file.

3. Block storage: The location where the optimized file data resides.

4. Additional free space: Optimized file and chunk storage is much smaller than the space used before optimization.

To take advantage of the deduplication technology in Windows Server 2012/R2, the environment must meet the following requirements:

1. A computer running Windows Server 2012/R2

2. A virtual machine containing at least one data volume;

OK, let's show you how to configure Data deduplication:

Login Server "HV-01", open Server Manager, click "Add Roles and Features":

In the Select Server roles location, expand file and storage services-tick "file and iSCSI Services"-tick "data deduplication duplicates" and click Next:

650) this.width=650; "height=" 457 "title=" clip_image005 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image005 " Src= "http://s3.51cto.com/wyfs02/M00/77/F1/wKiom1ZyBhTAF8C2AADNvVzoqHM257.jpg" border= "0"/>

To confirm that you have no problem installing the feature, click Install:

650) this.width=650; "height=" 458 "title=" clip_image007 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image007 " Src= "http://s3.51cto.com/wyfs02/M01/77/F0/wKioL1ZyBiDD4KaXAAC1Pad13m8484.jpg" border= "0"/>

After the installation is complete, click on the file file and storage service on the Server Manager page, click on the volume, and see the Dashboard to list data deduplication related information:

650) this.width=650; "height=" 411 "title=" clip_image009 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image009 " Src= "http://s3.51cto.com/wyfs02/M02/77/F0/wKioL1ZyBiPy4JwuAADQG8YrUuA396.jpg" border= "0"/>

Right-click the volume and select "Configure Data deduplication":

650) this.width=650; "height=" 301 "title=" clip_image011 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image011 " Src= "http://s3.51cto.com/wyfs02/M02/77/F0/wKioL1ZyBiXj8AUxAACRHgxVnQ0539.jpg" border= "0"/>

The Data Deduplication Setup Wizard will appear, where you can select "Disabled", "General purpose File Server" or "VDI server", and select "General purpose File Server" here:

650) this.width=650; "height=" 198 "title=" clip_image013 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image013 " Src= "http://s3.51cto.com/wyfs02/M02/77/F1/wKiom1ZyBhzj0HOQAABqrp8ehyI751.jpg" border= "0"/>

How long files can be set to perform data deduplication:

650) this.width=650; "height=" 104 "title=" clip_image014 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image014 " Src= "Http://s3.51cto.com/wyfs02/M00/77/F1/wKiom1ZyBh3hfXQUAABcSjJ3zYM944.png" border= "0"/>

If you do not want to make data deduplication for a file with a specific file name extension, you can choose which file extensions to exclude, such as the Word document that is commonly used in our work, which is the. doc format:

650) this.width=650; "height=" 138 "title=" clip_image015 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image015 " Src= "Http://s3.51cto.com/wyfs02/M01/77/F1/wKiom1ZyBh6zivJUAACCsUXToN0163.png" border= "0"/>

Data deduplication can not only exclude files that you want to delete as a file name extension, but also exclude folders for deduplication by specifying folders and subfolders, as shown in:

650) this.width=650; "height=" "title=" clip_image017 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image017 " Src= "http://s3.51cto.com/wyfs02/M02/77/F0/wKioL1ZyBimSXaVsAAA3i17roIY211.jpg" border= "0"/>

Set up a data deduplication schedule to set the job time based on the actual scenario:

650) this.width=650; "height=" 361 "title=" clip_image019 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image019 " Src= "http://s3.51cto.com/wyfs02/M00/77/F0/wKioL1ZyBiujfpB4AACln6WvoIk614.jpg" border= "0"/>

Set start time before data deduplication begins:

650) this.width=650; "height=" 369 "title=" clip_image021 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image021 " Src= "http://s3.51cto.com/wyfs02/M02/77/F1/wKiom1ZyBiOzqbNyAACi8KejIcs345.jpg" border= "0"/>

Click OK to wait for it to start performing data deduplication.

For data deduplication, it is important to note that data deduplication is not available in the system partition, which means that it can only be used on non-system partitions as shown:

650) this.width=650; "height=" 263 "title=" clip_image023 "style=" Border-top:0px;border-right:0px;background-image: none;border-bottom:0px;padding-top:0px;padding-left:0px;border-left:0px;padding-right:0px; "alt=" clip_image023 " Src= "http://s3.51cto.com/wyfs02/M01/77/F1/wKiom1ZyBiWTSecQAACMtOA03_s631.jpg" border= "0"/>

OK, here today's blog is over, more exciting content to look forward to the attention of everyone!

This article is from the "Wu Yuzhang Microsoft blog" blog, make sure to keep this source http://wuyvzhang.blog.51cto.com/9992636/1725464

Hyper-V Server data deduplication technology