What is a software-defined flash system?

In the age of software definition, there are many concepts that are easily confused. What is the difference between the software-defined flash system and the concepts mentioned in this article when customers often hear concepts like "software-defined storage", "Software-defined flash", etc. in the market? In particular, what is the difference between software-defined flash systems and software-defined flash? Here I try to share the essence of the software-defined flash system with you in a technical perspective.

at The China Cloud Computing Conference in the 5 month of this year, I made a report on "Key technologies for software-defined flash systems", in which we shared the key technologies behind the concept of "software-defined flash systems". Software-defined flash systems emphasize the storage of software on a common hardware platform, in a nutshell, a flash storage system is software. Software-defined flash systems are a large class in software-defined storage. Software-defined flash memory should be the concept mentioned in a Baidu paper that aims to build a NAND flash storage System based on the Host side software (SSD disk). It is more similar to the Host based SSD. A software-defined flash memory system is a way to build a flash storage system, such as a set of all-flash array systems in this way. On the basis of general hardware, a full-flash array is constructed by means of software-defined flash system.

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/71/86/wKioL1XTLCXhH7zaAAExuY870_8155.jpg "title=" 2.jpg " alt= "Wkiol1xtlcxhh7zaaaexuy870_8155.jpg"/>

A few years ago, due to hardware-level constraints, the storage was not able to meet the performance or data reliability requirements of the application on the basis of general-purpose hardware, such as:

1, in a disk storage system, if you need to achieve higher IOPS performance, you need to connect a large number of disks. a large number of disk connections require the support of a large number of HBA connection cards, and in order to meet this requirement, a large number of HBA cards need to be plugged into a single server . This is difficult to meet when the number of Pci/pcie bus slots is limited. Therefore, it is not easy to expand the IOPS performance of disk storage on a common hardware platform .

2, traditional disk storage systems require a large amount of memory as the data cache /cache, but the previous server did not provide a large amount of memory resources, need to be customized to improve the system, through the NUMA Way to implement a large memory system. As a result, datadomain systems such as EMC need to meet data deduplication needs through custom NUMA servers.

3, a few years ago if you need to achieve high availability of your storage system, you need to interconnect between nodes through a backend storage network. In order to achieve this interconnection requirement, it is often necessary to implement complex interconnection networks when designing a storage system, such as using RapidIO or IB for system interconnection. Support for specific hardware is required in the process of system interconnection. A few years ago in a server, if the system interconnect, a large number of disk connections and front-end IO Interconnection, then the need to occupy a large number of PCI resources. This system cannot be implemented with a common server and can only be achieved by customizing the hardware to meet storage system requirements.

4, the previous processor performance is relatively weak, in the process of data redundancy, if the CPU to complete the data codec, it is difficult to achieve very high IO performance. In order to achieve high performance, special hardware support is required. For example, using a hard raid card or a special ASIC chip to achieve the raid codec operation. Once this special hardware is used, special support is required for custom servers or software layers.

The development of server technology to today, an ordinaryX86the server can be easily implemented1TBthe memory capacity of the left and right; an ordinaryIntelprocessors can support12more than one of the physicalCpunuclear; an ordinaryIntelThe processor will be able to supportSse/avxinstruction set, easy to achieve the data codec; A common motherboard can support two-wayCpu, composedNumaarchitecture; An ordinary motherboard can export multiplePCIeSystem interconnect slots, each slot can support8Gbpsdata transmission rate; an ordinarySsdsstorage disks can easily reach tens of thousands ofIOPS; aSAS HBAon theSasinterface can achieve12Gbpsthe data rate, throughSasyou can build back-end storage networks. As a result, today's universal hardware is fully capable of meeting storage system requirements. Software-defined flash systems have good support for common hardware. In this case, the software-defined flash system simplifies the implementation of the system and becomes the inevitable development of the technology.

What's even more gratifying is that Intel 's CPUs are further developed in the direction of multicore and multi-core. The storage system itself is an IO -intensive application, and the development of multicore meets the requirements of storage technology. In addition,NVDIMM 's technology is becoming a server standard, which can further reduce the complexity of storage software technology and improve the reliability of data storage. For flash storage,the NVDIMM can just fill the performance gap between Flash SSD and memory, and compensate for the low-case SSD.

With strong support from server hardware, software-defined storage begins to land and can become the mainstream of storage systems. With the development of Flash SSD technology, all-flash systems will replace a large portion of the existing disk storage market in the future. It can be said that software-defined flash systems are a subclass of software-defined storage. Software-defined flash systems can achieve better performance and data reliability than software-defined storage, so they have a broader application base and real value. In all-flash storage, there is no denying that there will still be some hardware-customized all-flash systems, which may have higher performance in the future, and may achieve better data reliability at high performance. However, most products may be present in a software-defined form based on a common hardware platform (as shown).

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/71/89/wKiom1XTKmHwMH_6AACXg6JkWWc719.jpg "title=" 1.jpg " alt= "Wkiom1xtkmhwmh_6aacxg6jkwwc719.jpg"/>

Software-defined flash memory system is the inevitable development of technology, through the common hardware to build storage system platform, on this platform to deploy the core soul, through the Soul software to complete the storage function. This method can reduce the price, simplify the system development, and converge the product reliability. However, it is important to note that the general hardware is not as reliable as dedicated hardware, and how to build a reliable storage system on an unreliable hardware platform is a problem to be considered in system design. In this regard, software-defined flash systems require greater fault tolerance.

(from the store of Old Wu's blog-The way of storage)

This article is from the "Save the Way" blog, make sure to keep this source http://alanwu.blog.51cto.com/3652632/1685694

What is a software-defined flash system?