The quest for direct-connect technology for Windows Server 2016 storage spaces

about the direct connection of storage space technology, in my previous Blog Small and medium introduction, which is only about this distributed storage implementation of a small amount of extension, here this technology on my superficial research to interested friends to introduce. First, each node in the distributed storage is using local storage, but the logical layer is to ensure that each node can see the storage of all nodes, which sounds a bit strange, but in fact, the original storage vendors provide a similar structure of storage, Node head with local disk if the multi-head node is going to implement the disk to all the hook-up link of the head. The Windows Server 2016 storage Direct-connect technology we describe here is a software-defined storage category, so one of the first technologies to introduce is software storage bus technology.

Software storage bus technology abbreviation is a virtual storage bus spanning all storage cluster nodes, through which the nodes in all clusters can see all the local disks with fully meshed connections under all nodes. The storage bus contains two core components: the Clusport and Clusblft,clusport implement a virtual HBA interface, which enables the connection of the server disk devices under all cluster nodes. The CLUSBLFT implements the virtualization encapsulation of all the disk devices and headers connected through the Clusport; the storage component stack in Windows Server 2016 is as follows, where the green part is the two software storage bus components.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/76/A2/wKioL1ZYEeCDfucBAACXPd3nR7U201.png "/>

as we can see, the software storage bus communicates and communicates across the cluster nodes through SMB 3.0 and SMB Direct technology, providing a hierarchical, named instance of SMB on each cluster node host to provide the cluster shared volume file system used by the SMB consumer as shown in the upper layer to build reliability. Using the multi-channel and SMB Direct-connect technology provided by SMB 3.0 in the software storage bus, about SMB multichannel is described in my blog post , which can aggregate multiple links for improved overall bandwidth and transparent failover In addition, the core technology that is more direct to our cross-node connection is the SMB Direct Connect technology, the SMB direct connection utilizes the RDMA direct connection of the network card (IWARP, ROCE or InfiniBand), with this technology we can connect the disk device through the high bandwidth low latency network, and reduces the overhead of the CPU; you can simply imagine a high-speed virtual bus that is built from this direct-attached network to connect a cross-node device; Remember, two years ago, I was involved. Virtual Machine Online Migration demo with SMB Direct Connect technology on Microsoft technology Conference 2013 At the time, this technology was exploited in Windows Server R2.

Since it is called the software storage bus, then there must be the meaning of software-defined storage; The Software storage bus uses the fair connection algorithm to prevent a server from transitioning to disk, causing the other servers to be starved of disk access, and provides an IO priority optimization algorithm, For example, an application's IO priority (such as a virtual machine) due to system IO, of course, the algorithm must also ensure that the appropriate system IO needs to avoid the impact on the system, there is also a core optimization algorithm can be aggregated application (virtual machine) of the individual random IO for the aggregation of sequential IO to optimize the access speed of traditional mechanical hard disk.

Finally, the software storage bus provides cache optimizations for the storage bus cache in each cluster storage node, which is irrelevant to the upper-level virtual disks and storage pools, and belongs to the underlying optimization. The cache disk is a local SSD for the storage node, and the cache storage is redundant by cross-node mirroring, in contrast to the local high-capacity SATA mechanical hard drive as a data storage disk, where the local data storage disk caches the disk back-end locally on a RR polling basis, and if the cache node fails, the cache disk is redistributed. Depending on the local disk configuration, the cache policy selection will vary, as shown in the following table.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/76/A3/wKiom1ZYEYLyR881AAAv4-d9Z9U071.png "/>

Storage configuration	Cache device	Storage devices	Cache mode
SATA SSD + SATA HDD	All SATA SSDs	All SATA HDDs	Write
NVMe SSD + SATA HDD	All NVMe SSDs	All SATA HDDs	Write
NVMe SSD + SATA SSD	All NVMe SSDs	All SATA SSDs	Write

The cache device created by the software storage bus configuration reserves 32GB of storage space for the creation of metadata for the upper storage pool and virtual disks, and 10GB memory for each node per terabyte of data to be used for data structures , so in planning a hyper-converged architecture, it is necessary to consider that this part of the memory space reservation cannot be used as a virtual machine node workload.

The quest for direct-connect technology for Windows Server 2016 storage spaces