NET billion Xingyun Cblock about distributed data center implementation and optimization of Windows ODX based on distributed storage

1. Storage background for uninstall acceleration

Server virtualization is the mainstream of data center technology, large number of running virtual machines greatly improve the utilization of the server, the virtual machine batch creation, replication, migration greatly simplifies IT operations, but also brings a large number of data copy operations, which consumes the server's large amount of CPU, memory and network bandwidth resources.

For these issues, VMware proposes the Vaai (Vstorage API for Array Integration) interface, which wants to delegate some of the functionality of the VMware infrastructure to the storage array, by reducing the resource consumption of the data for virtual machines and physical servers, This improves the performance of your VMware infrastructure.

At the same time, Microsoft has added the ability to offload data transfer (offloaded data Transfer ODX) in Windows 8 or Windows Server 2012, hoping to offload the copy to storage hardware, reducing the overhead of virtualized servers.

This article will discuss the implementation and optimization of Windows ODX with you

With the use of storage hardware that supports offloading, ODX can perform file copy operations through the storage device without occupying the Hyper-V host CPU resources, and the storage device reads the data directly from one storage location and writes to another location. The design of offloaded data transfer conforms to the T10 XCOPY Lite specification, which provides an end-to-end design for data copy acceleration, theoretically supporting data replication between storage devices from different storage vendors.

2. Distributed Data Center cloud storage products for Xinghe Cloud Network

　　NET billion Xingyun Cblock is a "Wuhan Xing Yun Network Technology Co., Ltd." developed an innovative service-oriented products, this product is delivered in the form of a cabinet, it is a server, storage, network equipment, and system software, applications and management software, is a typical hyper-converged architecture. Cblcok from the perspective of management functions to provide users with "cloud Desktop Center", "Cloud Disaster Preparedness Center", "Cloud Resource Center" and "Cloud Management Center" four functions.
NET billion Xingyun Cblock is a data center resource engine that enables customers to have public cloud services and private cloud delivery capabilities in the context of legacy virtualization or private cloud. Cblock integrates the server, network, storage, security and management hardware and software resources of the data center, creatively provides virtual data center service, Cloud Disaster Recovery Center service, Cloud Desktop Center service, Cloud Management Center service four functions, make up the current market common cloud host, cloud Backup, The lack of functionality of public cloud resources such as cloud disks, and the ability to achieve dynamic scalability and full availability and reliability, has resulted in a qualitative leap in the resource management work function of the data center.

Overview of the 3.ODX implementation principle

ODX uses a token-based mechanism to replicate data inside or between smart storage arrays. The source and destination files to be copied can be on the same volume, on two different volumes managed by the same server, or on a cluster shared volume shared by multiple servers.

A token is a 512-byte random number that represents one or more extent (up to 128MB of actual data) and is typically part of a file or file. Tokens are generated by the storage controller through a hashing algorithm, while the token has a certain life cycle. So Microsoft has always considered security and compatibility to be a distinguishing feature of Windows ODX from VMware Vaai.

The performance of data replication is greatly improved by the fact that the token passed to the host represents the data, not the data itself. Because of the multiple command interactions in Odx, the Odx file copy requires at least 256KB files, and the overhead of too small files makes the xcopy operation meaningless.

Figure 1 Windows ODX Token-based copy operation

As shown, the main operational flow for data replication using ODX is as follows:

The user performs file copy/move through Windows Explorer or the command line.

Windows 8 or Windows Server 2012 sends a read request to the storage controller to perform a copy of the data (Populate_token command, which contains the source lba/Block length pair list)

The storage controller creates a token representing the data and sends it back to the host (SP creates pit token)

Host uses tokens to initiate write requests to the storage controller (Write_using_token command, containing a target lba/Block length pair list)

The storage controller verifies the validity of the token and the location of the source data based on the token information, and transmits the data to the target location.

Finally, the storage controller sends the results of ODX replication to the host (the host uses the Receive_rod_token_information command to query the results)

In order to implement the ODX feature, Odx adds several SCSI commands as follows:

The Populate_token command, which sends an unload read request to the storage controller, requests that a data token be generated for a piece of data on the virtual disk LUN a.

The Write_using_token command, which carries a data token to send an unload write request to the storage controller, requests that the data represented by the token be copied to the specified location.

The Receive_rod_token_information command sends an Unload query request to the storage controller, and the query unloads the read/write results.

4.ODX Optimization in distributed storage

4.1 Management and synchronization of tokens

For distributed storage clusters, multiple storage controllers are typically provided for concurrent access to IO and for high availability of IO paths (2). After Windows servers are connected through iSCSI MPIO and distributed storage, ODX may distribute Populate_token commands and Write_using_token commands to different storage controllers separately. This requires that a data token generated by one storage controller needs to be synchronized with other storage controls in real time. At the same time the token has a certain life cycle, so the storage controller needs to ensure the uniqueness of the data token, and periodically cleans up the data tokens that have expired.

Figure 2 Distributed storage cluster with multiple iSCSI server-side

4.2 Odx Related parameter tuning

When ODX is replicating data, a data fragment size of 64MB is transmitted by default, and the maximum data fragment that supports transmission is 256MB. Because of the different distributed storage environments, the number of HDD and SSD disks varies, so the most data transfer tile size varies in different environments. ODX supports the recommended optimal data slice size by storing the VPD description. Therefore, the optimal data transmission fragment size can be selected in combination with the configuration of distributed storage.

Considering that distributed storage in the production of data tokens, the need for the storage controller in real-time synchronization between the network, the latency is relatively large, so you can configure parameters to maximize the data token represented by the size of the data block, reduce the number of Populate_token command sent.

4.3 ODX-related flow control and some exception handling

When ODX is replicating data, it is usually read and written by large chunks, which puts a lot of pressure on the storage server's hard disk, which in turn affects the processing of other IO services. It is therefore necessary to make certain traffic restrictions on the IO associated with ODX when there are other business IO at the same time.

In some exception scenarios, the target location of the Write_using_token command and the location of the source file overlap, which may result in incorrect data processing, so you need to add some location checks before performing the write operation.

5.ODX Optimization Effect

The sandstone ups as an enterprise-class software-defined storage product provides the perfect support for Windows ODX with Odx,sandstone ups, which greatly saves the server's network bandwidth and CPU resources. Test data replication for a Windows 2012 server through a 1G network connection sandstone USP.

From the task Manager can see, when doing data replication, network bandwidth and memory, CPU consumption is not small, the characteristics of network bandwidth only occupied about 200K, but the data replication display speed averaged 200MB, far more than 1G network bandwidth limit. (Note: 200k is the direct bandwidth of Windows and storage controllers, Windows just sends control commands, and the actual data transfer is between storage networks.) )

Above, is our specific optimization practice on Windows ODX, in line with the open, pragmatic technology concept, the future, will actively absorb the industry's outstanding experience and technology products, thus creating a more perfect solution for the needs of users.

NET billion Xingyun Cblock about distributed data center implementation and optimization of Windows ODX based on distributed storage