Disk Storage Management

All operations on files in zookeeper involve disk access. Disk I/O speed and disk system reliability both directly affect system performance.
1. Data Organization and format:
Disk ---- disk surface ---- track ---- sector.
Low-level formatting-disk partitioning-advanced formatting
Note: Each partition is an independent Logical Disk.
2. Disk Type:
1) fixed head disk: There is a read/write head on each track, and all the heads are mounted in a rigid magnetic arm. These headers can access all tracks and perform parallel read/write operations, effectively improving the disk I/O speed.
2) Move head disk: Each disk is equipped with only one head and is loaded into the magnetic arm. To access all tracks on the disk, the head must be moved for seek. It can be seen that the mobile head can only read/write in serial mode, resulting in a slow I/O speed.
3. Disk access time
1) track time ts: the time it took to move the head to the specified track. The sum of the time s taken to start the magnetic arm and the time (TS = m * n + s) taken to move the N tracks of the head. M is a constant, which is related to the disk drive speed.
2) Rotation Delay Time TR: the time it takes to move the specified sector to the bottom of the head. In different disk types, the rotation speed is at least one order of magnitude different. Tr = 1/(2R), where R is the number of turns on the disk per second.
3) transmission time TT: refers to the time taken to read data from the disk or write data to the disk. The size of TT depends on the number of bytes read/write each time, and the rotation speed. Tt = B/(RN), where R is the number of turns on the disk per second; n is the number of bytes on a track, when the number of bytes for one read/write operation is equal to the number of bytes on the half track, TT and TR are the same.
Therefore, the access time is Ta = TS + Tr + TT.
4. disk scheduling
Because the time used to access the disk is mainly the seek time, the goal of disk scheduling is to minimize the average seek time of the disk. Currently, common disk scheduling algorithms include first-come service, shortest track time first, and scanning algorithms.
1) first-come first-served (FCFS): It schedules access to the disk based on the order of process requests. The advantage of this algorithm is that it is fair and simple, and requests of each process can be processed in sequence, so that requests of a process cannot be satisfied for a long time.
2) Shortest track time first (sstf): it requires that the accessed track be closest to the track where the current head is located to minimize the track time each time.
3) scan algorithm: This algorithm not only considers the distance between the target track and the current track, but also gives priority to the current direction of the head. For example, when the head is moving from the inside out, the next access object to be accessed by the scan algorithm should be the closest and closer to the current track. In this way, the magnetic arm is moved from the outbound to the outbound until no more magnetic channels need to be accessed.
4) Circular scanning algorithm (cscan): The cscan algorithm requires one-way moving of the head. For example, the head is only moved from the inside out. After the head is moved to the outermost track and accessed, the head immediately returns to the path that the Shide wants to access. It also forms a cycle with the smallest track number followed by the largest track number for cyclic scanning.
5) nstepscan algorithm: the n-step scan algorithm divides the disk Request queue into several sub-Queues with N length. The disk scheduling will process these sub-queues in sequence using the FCFS algorithm. Each processing of a sub-queue is based on the scan algorithm. After processing a queue, other queues are processed. When a sub-queue is being processed, if a new disk I/O Request occurs, the new request process is put into another queue, which may avoid adhesion.
6) fscan algorithm: only the disk Request queue is divided into two subqueues. One is a queue formed by all the processes requesting disk I/O, and is processed by the scan algorithm by disk scheduling. During the scan, all new disk I/O Request processes are put into another request queue waiting for processing.
5. High-speed disk cache
1) format: the storage space in the memory is used to store information in a series of disk blocks read from the disk.
2) data delivery method:
Data delivery: This directly transfers the data in the cache to the memory of the requester process.
Pointer delivery: This only delivers the pointer pointing to a certain area of the cache to the requester process.
6. other methods to increase disk I/O speed
1) read ahead
2) Delayed write
3) Optimize the distribution of physical Blocks
4) Virtual Disk
7. low-cost redundant disk array (RAID): it uses a disk array controller to centrally manage and control a group of disk drives (from several to dozens, A highly reliable and fast large-capacity disk system.
To speed up the orientation of disks, the cross-Access Technology Applied in large and medium computers has been applied to the disk storage system. In this system, there are multiple disk drives. The system divides the data in each disk into several sub-disks, then, store the data of each sub-disk block to the same location in different disks. In the future, when you want to transmit data from a disk block to the memory, use parallel transmission to transmit data from the sub-disk blocks to the memory at the same time, therefore, the transmission time is greatly reduced.
Raid classification: RAID 0 ~ Raid 7
RAID 0: only parallel cross-access is provided, and there is no redundancy check function, resulting in poor disk system reliability.
Raid 1: Provides Disk Mirroring. Exchange disk capacity for access speed.
RAID 3: A disk array with parallel transmission functions. A parity disk is used to verify data. Reduces the number of redundant disks required. It is used for scientific computing and image processing.
RAID 5: A disk array with independent transmission functions. Each drive has its own independent data path, read/write independently, and no dedicated verification disk. The verification information used for error correction is spiral distributed on all data disks. It is used to process transactions with frequent I/O operations.