Computer knowledge Supplements (iii) mechanism of understanding disk

Disk is an important memory, located under the main memory structure, is a permanent storage of media. Knowledge supplements (i) Understanding virtual memory mechanism at the bottom of the computerThis article says that virtual memory is disk-oriented, and understanding how the disk works is a great help in understanding many of the concepts of computers. Especially in the field of database and distributed storage, you should always deal with disk.

This piece of disk has several main concepts:

1. Basic structure and working principle of disk

2. How to efficiently exchange data with memory under the virtual memory mechanism

3. How the disk guarantees the reliability of data storage and the failure recovery

Basic structure and working principle of disk

From a single disk, consisting of concentric circles, a concentric circle is a track, each track is composed of multiple sectors, each sector is separated by a non-magnetic gap. The sector has a magnetic substance that supports both read and write operations .

the number of sectors per track is a constant, and the size of each sector is typically 4KB. A sector is a basic physical unit of disk .

Each disc corresponds to a transmission arm, and the head of the transmission arm has a read/write head that can read and write disk media. The drive arm can cover all the tracks and can access all sectors of the disk by rotating the disc face and moving the drive arm.

Ordinary hard disk has a plurality of disk surface composition, each disk surface has the above structure. Multiple drive arms on a hard drive are fixed together, that is, the radius of the move is the same. So that the various tracks at the same time at multiple heads constitute a cylinder, the advantage of using multiple disks is that the data of the same cylinder can be read simultaneously.

We know that storage systems use logical unit blocks to represent basic units of data, which can increase the efficiency of storage, such as virtual memory using pages as the basic storage unit. Also in the disk domain, the block is used to manage the disk. The sector is the basic physical unit with a size of 4KB. Disk blocks are generally 4kb-64kb and contain one or more sectors. Main memory and disk exchange data with blocks as the basic unit.

Each disk has a disk controller to manage one or more disks. The role of the disk processor is:

1. Control the drive arm, positioning the head to a specific radius position

2. Select a disk that is ready to read and write, and navigate to a specific sector

3. Transfer binary data read from the required sector to main memory, or write the data of main memory back to the sector of the disk

To see an actual disk parameter, the Megatron 747 disk has the following parameters:

1.8 discs, 16 discs

2. Each disc has a 2^16 track

3. An average of 256 sectors per track

4.4KB bytes per sector

So it is 16 disks * 2^16 TRACKS * 256 sectors * 4KB bytes = 1TB in size. A track holds 1MB bytes. If a block is 16KB, then 1 blocks use 4 contiguous sectors, with 32 blocks on one track

There are 3 steps to read and write a track, i.e. seek time + rotation delay + transmission time

1. Disk controller sets the head combination to the desired seek time on the cylinder of the track on which the disk block is located

2. Disk controller waits for the first sector of the access block to go to the head, that is, the rotation delay

3. When the disk controller reads and writes data, the gap in the sector and sector of the data is passed through the head, i.e. the transfer time

The seek time depends on the distance from the head to its location, and if the head is just in the cylinder to be accessed, the seek time is 0, but it takes 1ms of time to start the head. The head takes 10ms of time to pass all the tracks, so seek time in 0-10ms, average 5ms

The disk rotates around 10ms, so the rotation delay is 0-10ms, averaging 5ms

The transmission time is relatively small, in milliseconds.

So the average latency for reading 1 bytes of disk is 10ms, and the maximum latency is almost twice times that of it.

It is natural to think that increasing the disk read and write speed is to minimize the seek time and rotation delay, such as the following methods:

1. Organize data by cylinder, the block to be accessed together on the same cylinder, which often avoids seek time, may also avoid rotation delay

2. Store data separately on a number of relatively small disks rather than on a large disk, which allows more head group devices to access disk blocks individually, increasing the amount of disk block access per unit of time

3. Mirrored disk, two or more copies of the data on different disks, on the one hand to ensure redundant storage of data, on the other hand, we can access multiple disk blocks

4. The continuous fast preload into the main memory buffer, which is the use of spatial locality, from the point of view of caching principle to speed up disk access

Understanding the data exchange between disk and main memory from the perspective of virtual RAM

First understand how the CPU is going to request data from disk IO. We know that the various components of the computer are connected by bus, including data bus, address bus, control bus and so on. The CPU uses memory-mapped IO technology to access the IO device. The virtual address space has a dedicated address, called an IO port, to communicate with the IO device, and when a device is connected to the bus, it is mapped to one or more ports.

Assuming that the disk controller is mapped to the 0xa0 port, the CPU initiates IO read process as follows:

1. The CPU writes the command, the logical block number, the destination virtual memory address to the 0XA0 port, and initiates a disk read request

2. The disk controller translates the logical block number into the corresponding sector location read sector, which performs disk data-to-memory transfers by DMA (direct memory access, directly memery access) controller, without CPU involvement

3. When the DMA transfer is complete, the disk controller interrupts to notify the CPU that the read is complete

There are a few points to note:

1. The CPU only initiates read requests and ends with interrupt-aware read operations, and other tasks can be performed at other times to improve CPU utilization

2. Memory generally has a buffer to disk interaction, which is actually the use of the principle of caching, in memory to open up a buffer, the actual memory read-write and buffer interaction, buffer and disk interaction, so as to improve the efficiency of memory read and write

3. Memory and disk transmission data in the smallest units, this is also the application of the principle of caching, one or more blocks at a time.

From the point of view of virtual memory, the virtual pages and blocks in the disk are mapped, and when the virtual pages are loaded into the physical pages of memory, the disk blocks corresponding to the virtual memory are loaded into the physical pages of the corresponding addresses of the memory by DMA. When a physical page is written back to disk, it is also transmitted by DMA to the disk controller, which is written by the disk controller to the sector corresponding to the disk block. Memory and disk Exchange data when the actual use of memory buffers to speed up the disk access speed.

The purpose of the buffer is to fit two devices with inconsistent speed, from how the disk works we see that the disk operation is a very slow operation, the memory operation is a very fast operation compared to the disk operation, in order to let the memory on the disk read and write do not have to wait for the disk operation return and return, The operating system sets up a memory buffer to speed up access to the disk.

Memory buffers are the use of caching principles, which are an important component of transferring data between disk and memory. A separate article is written about the principle of memory buffers. To put it simply, the memory buffers consist of cache blocks, and the size of the cache block is the same as the size of the disk block. Each cache block has a buffer_head data structure that stores the DeviceID and disk blocks of the disk that corresponds to the cache block, which is equivalent to binding a cache block and a physical disk block. The exchange of data between the disk and the memory buffer is in blocks as the base unit.

With a memory buffer, the CPU accesses some data from a disk file, and only needs to provide the disk block number where the data is located, and it can find out if the contents of the disk block have been cached from the memory buffer.

In addition, the disk is specifically set up a swap area swap to hold the page swapped out from memory, swap is related to the operating system's page recycling subsystem, and the memory buffer is not directly associated with, there will be a separate write on the disk swap zone swap mechanism.

How the disk guarantees data reliability and failure recovery

Disk guarantees that the reliability of data storage is mainly by redundancy. Redundancy has many strategies, such as

1. A stable storage of a single disk, the sector in accordance with 22 pairs, such as x and y pairs, each time the write x and y to write the same data, so when one of the failure, you can use another piece to recover. Two sectors fail at the same time in a very small situation

2. Disk array consisting of multiple disks raid,raid0 is the way of using redundant images

3. Software-guaranteed redundancy, such as HDFS with 3 redundant storage policies

Disk failures usually have several

1. Intermittent failure, one attempt to read or write a sector was unsuccessful, but after repeated attempts to read and write successfully

2. Media corruption, one or more bits of disk permanently damaged, resulting in read impossible to read some sectors

3. Write fault, attempt to write a sector, that is, cannot write correctly, and can not retrieve the previously written sector, may be written when the power interruption caused by

4. Disk crashes, the entire disk is permanently unreadable

For intermittent failures, the use of parity, you can quickly check the success of a read and write. The disk controller checks read and write correctly by parity, fails to retry, and returns to read and write failures if the number of attempts is exceeded

For some bits in the disk corruption, or write failure, you can use the above-mentioned single-disk stable storage method, the sector pair storage, so as to reduce the impact of media corruption and write failure

For the entire disk crash, you can use a hardware-based RAID disk array, or the software guarantees redundant storage of multiple disks, to deal with the impact of the entire disk crash, and can quickly recover the damaged disk data

Computer knowledge Supplements (iii) mechanism of understanding disk