Analysis of the relationship between the I/O process and storage performance

Analysis of the relationship between the I/O process and storage performance

https://community.emc.com/docs/DOC-28653

Performance "The term can be said to accompany the development of the entire IT industry, every time new technologies emerge, from hardware to software in most cases are around performance improvement. The Moore theorem points out that the CPU's processing speed doubles every 18 months, but it seems to be slowing down in the second decade of the 21st century. But industry leaders in the IT industry continue to challenge physical limits by constantly seeking breakthroughs in computer performance. A closer look at the storage industry, the rollout of every new storage product, is also centered around how faster and better service front-end server I/O requests are. This paper introduces the flow of I/O (Block) and attempts to understand the connection between the entire I/O stream and the storage performance. This article serves as a storage-based introductory article to help readers understand the seemingly simple data read and write in more detail.

More information

Storage I/O streaming vs. storage performance:

The process of storing I/O (hereafter referred to as I/O) is the process by which the computer reads data and writes data on the memory. This memory can be either non-persistent storage (RAM) or a persistent storage similar to a hard disk. A complete I/O can be understood as a data unit that completes the bidirectional process from the initiator to the receiving end. In an enterprise-class storage environment, this process passes through multiple nodes, and each node uses a different data transfer protocol. The transfer of a complete I/o between each of the different nodes may be split into multiple I/O, then transferred from one node to another, and then the same process goes back to the source side.

Demonstrates the changes that a file makes to each node through the entire I/O path (in the EMC Symmetrix Storage Array example):

The entire I/O stream goes through several nodes:

• The file system– file system divides the files into blocks using File System Manager, based on the mapping of the files to the block, requesting the HBA to be sent.
• Hba–hba performs an operation on this series of smaller units of work, converting this portion of I/O to the Fibre Channel protocol, wrapping the frame to a maximum of 2KB to the next connection node FC Switch.
• FC SWITCH–FC Switch sends these frames to the front-end port (Front Adapter) of the storage system via the FC fabric network.
• The Storage fa– storage front-end port will reseal these FC frames to match the HBA's initial send I/O, then FA will transfer the data to the array cache (Storage array caches)
• Storage array cache– array cache processing I/O is usually done in two situations: 1. Directly returns the signal that the data has been written to the HBA, which is called write-back and is also the way most storage arrays are processed. 2. The data is written to the cache and then flushed to the physical disk, called write-through. After I/O is stored in the cache, it is left to the backend controller (Disk Adapter) to continue processing and then returns the signal that the data has written to the HBA.
• Disk adapter– Both methods, and finally writes I/O to the physical disk. This process is controlled by the backend disk adapter, which, depending on the raid level of the back-end physical disk, becomes two or more actual I/O.

According to the I/O flow above, a complete I/O transfer, elapsed time of the node can be summarized as follows:

• Cpu–ram, complete the operation of the host file system to the HBA.
• HBA–FA, complete the transmission process in the optical fiber network.
• Fa–cache that stores the time that the front-end card writes data to the cache.
• Da–drive, stores the time at which the back-end card writes data from the cache to the physical disk.

In the table below, a comparison of data access times in different stages is made, and a 8KB I/O completes the approximate time-consuming flow of the I/O. (The time spent in the table is obtained by dividing the transmit data per second, such as HBA to FA at a speed of 102,400kb/seconds divided by 8KB to get 78μs). Depending on the data in the table, it is clear that the I/O from the host's file system began to transmit to the storage array cache at a small percentage of this I/O, due to the limitations of the mechanical hard disk, the maximum time spent on DA to the physical disk. If you use a flash drive, that data will be significantly reduced, but it is still relatively large compared to the transfer time of several other nodes.

Transmission of 8KB I/O	Time-consuming (unit: μ s)	Expand 1,000,000 time consuming
CPU-to-RAM transmission, 166MHz 64-bit bus	6μs	6 sec
HBA to fa,100mb/sec rate	78μs	1 min. sec
FA to cache (Symmetrix DMX Director 2.5GHz)	26μs	Sec
Da to Physical disk, 73GB 15,000 rpm, Seek 3.6ms,latency 2.0ms	5,700μs	1 hr min

As you can see, the role of the storage array cache in the entire I/O stream is critical. The processing efficiency and size of the cache directly affect the speed of I/O processing. However, in the actual environment, even if the storage array cache work properly, the host's I/O will not reach the level of 100μs that is 0.1ms, usually around 1-3ms, I think I/O processing in a relatively high performance mode. The reason is because the other two factors are "Data header processing" and "concurrency."

1. "Header processing" because the data of each I/O in the I/O stream does not consist of data only, as shown, an I/O contains negotiation,acknowledgement for each node in the I/O stream to be transferred and managed in addition to the data. It contains the same "handshaking" information as the TCP/IP, as well as the flow control information, such as the initialization of the transmission, the end of the communication and so on. The header will define some information, such as CRC checksum, to ensure consistency of the data. The processing of all this data consumes a certain amount of processing resources and increases the time-consuming of I/O streams.

2. "Concurrency". Because the I/O stream is not possible to process only one I/O at a time, all I/O is in a large number of concurrent cases in the HBA,FC,FA and DA processes. The main time-consuming depends on the wait of the I/O queue, although the storage array is optimized for concurrency. The same processing of slice is done in a queue form. As shown, when storage faces multiple I/O processing at the same time, there is always an I/O that will come out at the end of the entire stream and increase I/O time. So, at each node of the I/O stream, there is a bottleneck, or a short board. I/O time-consuming will increase.

In summary, the relationship between I/O flow and storage performance can be summed up in the following points:

• Complete an I/O stream the main experienced node has HBA,FC network, storage front port FA, storage cache, post storage port, physical disk. The most time-consuming part of a process is a physical disk.
• The size and processing of the storage array's cache directly affects the performance of the I/O stream and is one of the important indicators for defining a storage array.
• The processing speed of I/O is usually far away from the theoretical value, resulting in a queue delay due to the large number of concurrent volumes.
• The way to optimize I/O can start with multiple nodes, and the most noticeable effect is to increase the speed of the physical disk. Because the storage array puts as much data into the cache as possible, when the cache is full, the data exchange is entirely dependent on the speed of the physical disk.
• Appropriate RAID levels are chosen, because the read and write ratios of different RAID levels vary greatly, which can result in several times more time consuming physical disk processing. Reference: About RAID write penalty (write penalty) and IOPS calculation

Analysis of the relationship between the I/O process and storage performance