Discussion on the application of NVDIMM in flash storage

Ssdsas a new type of storage media exposed to a common block device, the traditional application seems to be without any changes in theSsdsrun on. In the actual application process, the traditional business can indeedSsdsrun directly on it, but the problem isSsdswas not fully exploited, and the advantages were not fully exerted;Ioattribute can causeSsdsa new problem arises. For example, in some application sites, users findSsdsThe service life is quickly exhausted, the writing amplification system becomes very large, the service life is different from the expectation. The manufacturer's write amplification system is in a specificIO PatternThe actual application due to the existence of a large number of512byte lowercase problem, the data distribution is not "perfect", causingSsdsthe internal write amplification factor is larger than expected, thus affectingSsdsService life. In a word,Ssdsthe use of the business is not so simple as itsIO Patternclosely related. If you want to have an enterprise-classSsdsperformance, consistency, and service life, you need to targetSsdsthe storage software stack is refactored. One idea of the refactoring process is to optimize the inputSsdsof theIO Pattern, to getSsdsthe best working state to playSsdsthe performance, avoidSsdsthe problem.

in theIO Patternof the optimization method, is boomingNVDIMMneed special attention. So to speak, technically.NVDIMMand theSsdsis a natural pair, they can be characteristic complementary, in the short term coexistence development. NVDIMMis not a revolutionary, new thing, many disk storage systems have long been usedNVRAMtechnology to achieve a power-down nonvolatile cache. ThisNVRAMbased onPCIebus to ensure the reliability of the system in case of power loss. NVRAMIt is typically used as a data cache in the storage system. To ensure data reliability, two pieces are designed in one systemNVRAM, throughMirrorredundant data in the same way. NVDIMMwith theNVRAMThe biggest difference is that the interface isPCIetransferred to theDimm(Memory interface), followed by the power-down data protectionNAND Flashcombined with supercapacitors to ensure that the data is not lost, or directly using a new type of storage media, such asXpoint,ReRAMand so on.

in a hierarchical storage system, The performance of the NVDIMM is essentially the same as that of DDR memory. The performance is far more than the SSD, its IO access Latency is around dozens of to hundreds of nanoseconds, and SSD access Latency has reached more than hundred subtle. However,SSDs are much higher in capacity than NVDIMM. Therefore, in the existing tiered storage architecture, the introduction of NVDIMM can be very good for SSD, disk storage optimization, especially the IO Pattern optimization.

650) this.width=650; "src=" http://s4.51cto.com/wyfs02/M02/8B/CC/wKioL1hZLtfxqighAACSznV-DR8876.jpg "style=" float: none; "title=" 1.jpg "alt=" Wkiol1hzltfxqighaacsznv-dr8876.jpg "/>

2014 The first time you start using Nvdimm nvdimm The standard is just starting from snia< Span style= "font-family: ' The song Body '; > Organizational development. At that time the nvdimm SLC NAND + SDRAM + supercap design method, the principle of work is basically the same, this Nvdimm nvdimm-n year and 2016 Extend the other two Nvdimm product standard, respectively nvdimm-f and nvdimm-p< Span style= "font-family: ' The song Body '; The working principle and morphology of these products will be different. It can be seen from the nvdimm The evolution direction of product standards. Different nvdimm The product form is nvdimm-n

650) this.width=650; "src=" http://s1.51cto.com/wyfs02/M00/8B/CF/wKiom1hZLtjzNx8SAAEXxOkPsHo302.jpg "style=" float: none; "title=" 2.jpg "alt=" Wkiom1hzltjznx8saaexxokpsho302.jpg "/>

in the existing standard system, the main product form hasNvdimm-n,Nvdimm-fas wellNvdimm-p, as shown in the respective features. WhichNvdimm-nis the earliest form of product, used in storage to replaceNVRAMapplications, which are accessed in the system by means of a visit. On the performanceNvdimm-nand the ordinaryDdrThe memory is the same, the difference is that when the system is out of power,Nvdimm-nis synchronously written to the memory strip.SLC NAND Flashthe. The entire power-down process is powered by a super capacitor. Nvdimm-fis toFlashmadeDimminterface to access the block device through the block devices interface. Some storage companies are doing such products, which can be further avoided byPCIeBus-introducedSsdsaccess delay. As directly throughDimmInterface AccessNAND Flash, So,Nvdimm-fThere is no way of performance withNvdimm-ncompared. By adoptingNvdimm-fMorphology ofSsds, you can build high-density flash storage systems, but the disadvantage is that you need to customize the server platform. The latest proposedNvdimm-pis a comprehensiveNvdimm-nand theNvdimm-fof the product form, the technical structure of the above two are different, the biggest idea is mixed memory andNAND Flash, and provides a block device and memory access interface, which is also a performance-Nvdimm-nand theNvdimm-fbetween, the capacity is much larger thanNvdimm-n, can do andNvdimm-fthe same storage capacity.

650) this.width=650; "src=" http://s1.51cto.com/wyfs02/M00/8B/CF/wKiom1hZLtiCb2vzAAESO7UGuCE358.jpg "style=" float: none; "title=" 3.jpg "alt=" Wkiom1hzlticb2vzaaeso7uguce358.jpg "/>

Nvdimm-n is the first product form, the default is that the NVDIMM is this product form. The principle of nvdimm-n is as follows:

650) this.width=650; "src=" http://s4.51cto.com/wyfs02/M00/8B/CC/wKioL1hZLtnjZdFFAADTOTv2pl8744.jpg "style=" float: none; "title=" 4.jpg "alt=" Wkiol1hzltnjzdffaadtotv2pl8744.jpg "/>

as you can see, the memory bus passes throughBufferswitches can be accessed directlyDram, so, without introducing additionalIodelay. In the normal use of the process,Nvdimm-nin the system is a piece of normal memory. In this subsystem we can find that there is a callCntlrController, early product the controller is usedFpga, the controller can be used to persist the memory data and load the operation. When the controller receivesSAVEafter the command signal, theDramSave the data in theNandafter the system re-powerNandload data into theDramthe. Because of the function of this controller, it is ensured that in the case of abnormal power loss in the systemDramdata is not lost. and theNvdimm-pthe operating system cannot directly accessNanddata in the. ForNvdimm-n, one of the most important technical points is how to trigger the persistent operation of the memory data in the case of power-down. In the earlyNvdimm-nin the design, special motherboard support is usually required. There is a hardware unit in this type of motherboard, for example withCpldthe power supply is monitored, and when the system is found to be lower than expected, the processor starts to trigger power-down protection. In this type of design, there is a driver for the operating system that is responsible for the system power-downCpuData refresh operations. By this driver , the processorCacherefreshes the contents of theNVDIMM, and then toNVDIMMThe hardware sends the data to save the signal. NVDIMMreceived aSAVEafter the signal isSdrampersist the data in the. The entire process can be described as shown in the following:

650) this.width=650; "src=" http://s4.51cto.com/wyfs02/M01/8B/CF/wKiom1hZLtrTjyXHAADwee_jv1w164.jpg "style=" float: none; "title=" 5.jpg "alt=" Wkiom1hzltrtjyxhaadwee_jv1w164.jpg "/>

one of the biggest problems with this mode of operation is that it is not universal, requires a hardware platform customization, and also requires a large capacitance of the endurance, to ensure that the processor all data in the Cache is flushed to memory. Another more general approach is the adoption of Intel 's standard ADR technology , which, when an Intel processor detects a power outage, sends a save to the NVDIMM signal, and let the NVDIMM is in a self-refreshing mode of operation. The biggest problem in this way is how to ensure that the processor caches the data of the power-off non-volatile. The previous two years have discussed this issue with the NVDIMM standards Organization, and there is no standard to support this feature. In other words, the data in the processor Cache is not guaranteed to be non-volatile in the event of an abnormal system power-down. This is a serious flaw for storage applications and is likely to be a problem with data loss in the event of a system power-down. Therefore, in the application of NVDIMM must solve this problem, this is NVDIMM in the storage application of the design focus.

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/8B/CC/wKioL1hZLtqy5KdTAAD4dw3CBmQ911.jpg "style=" float: none; "title=" 6.jpg "alt=" Wkiol1hzltqy5kdtaad4dw3cbmq911.jpg "/>

nearly two years Nvdimm-n has been standardized by the relevant organizations, and ADR technology has become the mainstream technical means. A new interface signal is also defined in the DIMM interface. the JEDEC JC45.6 standard standardizes 12V power supply,save_n signals,event# signals, and the electrical characteristics of i²c, as shown in.

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/8B/CC/wKioL1hZLtvAPKZ4AACwi-416Os492.jpg "style=" float: none; "title=" 7.jpg "alt=" Wkiol1hzltvapkz4aacwi-416os492.jpg "/>

in the Nvdimm-n power-down protection is a major contributor to the super-capacitor, which has a high capacitance density, with fast charging characteristics, in the system power-down can provide NVDIMM with enough energy to ensure that the SDRAM The data in the NAND Flash . There seems to be nothing special about this component and it seems that there is no need to think too much about it in a storage system. In fact, any component can fail, if the supercapacitor fails, then the system will be lost in the event of data loss disaster. The lifetime of the supercapacitor is closely related to the operating temperature, and the higher the temperature, the shorter the lifespan of the supercapacitor, as shown in:

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/8B/CF/wKiom1hZLtuA8dsXAADLRR-p1JA165.jpg "style=" float: none; "title=" 8.jpg "alt=" Wkiom1hzltua8dsxaadlrr-p1ja165.jpg "/>

it can be seen from this graph that the lifetime of supercapacitors decreases with the increase of ambient operating temperature in the case of different capacitance output voltages. In the operating environment, many supercapacitors have a lifespan of less than 5 years. in SSD storage systems,the operating temperature of the NVDIMM environment is normal, so the service life of supercapacitors is a problem used in storage applications. In order to solve this problem, in the storage system design process, need to monitor the service life and status of supercapacitors, if there are any trouble, need early warning, otherwise it will lead to data disaster. So, after introducing NVDIMM into the storage system, we found thatNVDIMM solves many problems in the storage System , and can mate with SSDs to optimize IO Pattern, But at the same time, a lot of difficult practical problems have been introduced.

personally feel rightNvdimm-nThe larger product of impact comparison is the new storage medium, for exampleXpointrepresents the semiconductor storage medium. XpointComposition ofMemoryThe biggest benefit is that faulty parts such as supercapacitors are not required, although performance is not asNvdimm-n, but in a storage application, you need to resolve the processorCachethe problem, so, combined down the performanceXpointThere is not necessarily a big gap. Xpointfrom2015No products have been released since the year, but the use ofXpointBuild thePCIe SSDalready has the test result, the performance far exceeds the adoptionNAND FlashBuild theSsds. As shown, the queue depth is1case, the use ofXpointBuild theSsdscan be run to the near8millionIOPS, is currentlyPCIe P3700 SSDof the7.32times. In the future, this new type of storage media can beNVDIMMinnovation on the product form, andNAND Flashof theSsdstogether, Kone creates high-performance, efficient, and reliable data storage systems.

650) this.width=650; "src=" http://s5.51cto.com/wyfs02/M02/8B/CC/wKioL1hZLtyhJdqxAADcsJ01avA268.jpg "style=" float: none; "title=" 9.jpg "alt=" Wkiol1hzltyhjdqxaadcsj01ava268.jpg "/>

The Nvdimm-f is a relatively straightforward product form that connects NAND Flash directly to the memory bus via the controller . The exact principle is as shown. In the system, the data in the NAND is accessed by means of a block device . compared to the PCIe bus- based SSD , the data access bus has changed and has a higher bus bandwidth. in the controller that controls the NAND , you need to implement an FTL similar to the SSD that wrapsthe Nand into a block device for system use.

650) this.width=650; "src=" http://s5.51cto.com/wyfs02/M01/8B/CF/wKiom1hZLtzD9FwxAACi8MF6d7Q943.jpg "style=" float: none; "title=" 10.jpg "alt=" Wkiom1hzltzd9fwxaaci8mf6d7q943.jpg "/>

in the storage system, some vendors adopt this design idea. By customizing the hardware platform, a large number of NVDIMM-F modules are integrated on the memory bus to build high-density storage within a small physical space. in the early days of a Skyra storage company that used this design idea to create petabytes of storage in 1U of physical space , the idea was still a bit crazy in that era. With the increase of the storage density of NAND Flash particles, it is not difficult to achieve petabyte storage in 1U space , and it is not necessary to adopt physical form such as nvdimm-f.

and the Compared to Nvdimm-n ,nvdimm-f 's access performance is relatively low, which is mainly limited to the performance of NAND Flash. If you replace NAND Flash with media such as Xpoint, the access performance will be greatly improved. In order to balance the performance and capacity between memory and NAND, the Nvdimm-p product form was born, and the principle structure of the product was different from the first two, which connected DRAM and NAND by means of the controller. , and both NAND and DRAM space can be exposed to the system. This approach can be understood as a hybrid storage architecture that flushes data from DRAM to NAND in the event of a sudden system power-down, using the SAVE signal .

650) this.width=650; "src=" http://s4.51cto.com/wyfs02/M00/8B/CF/wKiom1hZLt2hjrv1AAC0kcd4Xhk136.jpg "style=" float: none; "title=" 11.jpg "alt=" Wkiom1hzlt2hjrv1aac0kcd4xhk136.jpg "/>

with the This form can do a lot of storage capacity compared to nvdimm-n. However, with the addition of additional controller units between the DRAM and memory controllers, the IO access performance of DRAM is affected, and performance is also between nvdimm-n and Between the Nvdimm-f . Because it is a hybrid storage mode, it is more worrying about the jitter of performance.

to fully optimize SSD Flash storage System,NVDIMM is a very good storage medium, through the NVDIMM can realize the SSD IO Pattern optimization processing. But nothing is perfect, the use of NVDIMM in a storage system does not find a lifeline, and Many new problems are encountered while using NVDIMM's fine features. The new type of storage architecture design process needs to take advantage of the benefits of new media, and more importantly, it needs to solve the problem of new media introduction. We look forward to the further development and improvement of NVDIMM technology and play a more important role in future high-performance flash storage systems.

(The way of storage)

This article is from the "Save the Way" blog, make sure to keep this source http://alanwu.blog.51cto.com/3652632/1884497

Discussion on the application of NVDIMM in flash storage