Analysis of RAID 10 data disk storage in Linux kernel MD module and interpretation of several key parameters

Keywords: Linux software raid, MD, raid10, r10_private_data_s, near_copies, far_copies, far_offset

Kernel version: 2.6.18

The raid 10 implemented by the Linux kernel MD module has three laout modes: near, far, and offset. When you use the mdadm command to create a raid 10 disk, you can choose one of the three types (in fact, the near and far modes can be mixed, but it makes no sense to make four or more backups of one chunk ).

[1] features of three modes

In the near mode, the same Chunk is copied n copies, usually two copies. The two copies will have the same offset on two adjacent disks. In the far mode, N copies of the same chunk will be separated from each other. The first copy of all data will exist in front of all disks in the form of RAID 0 strip, and the second copy of all data will also exist in the form of RAID 0 strip, at the same time, the far mode ensures that all backups of the same chunk are stored on different disks.

The far method can provide continuous read performance equivalent to raid0, but the write performance will be reduced (it will bring more seek time for writing) [1] [2], therefore, the far method is more suitable for read and write scenarios.

In the offset mode, N copies of the same chunk will exist on a continuous disk at a continuous offset (one disk offset and one chunk offset ). Copies of each stripe are stored at the offset of one disk. If the chunk size is appropriate, the offset method can provide the same read performance as the far method, but the write performance is not as bad as the near method.

The number of RAID 10 disks does not have to be an integer multiple of the number of chunk copies, but at least there must be a disk with so many chunk copies.

[2] example of three layout Methods

(Columns are drives, rows are stripes, letters are chunks of data)

N2 layout:

2 drives 3 drives 4 drives
--------------------------------
A1 A1 A1 A1 A2 A1 A2 A2 A2
A2 A2 A2 A3 A3 A3 A4 A4
A3 A3 A4 A4 A5 A5 A6 A6
A4 A4 A5 A6 A6 A7 A7 A8
..................

F2 layout:

2 drives 3 drives 4 drives
------------------------------------------
A1 A2 A1 A2 A3 A1 A2 A3 A4
A3 A4 A4 A5 A6 A5 A6 A7 A8
A5 A6 A7 A8 A9 A10 A11 A12
..................
A2 A1 A3 A1 A2 A4 A1 A2 A3
A4 A3 A6 A4 A5 A5 A6 A7
A6 A5 A9 A7 A8 A12 A9 A10 A11
..................

N2 F2 layout:

A1 A1 A2 A1 A1 A2 A2 A3
A3 A3 A4 A3 A4 A4 A4 A5 A5
A5 A5 A6 A6 A6 A6 A7 A7 A8
A7 A7 A8 A8 A9 A9 A10 A10
..................
A2 A2 A1 A1 A2 A3 A1 A2
A4 A4 A3 A5 A5 A3 A4 A4
A6 A6 A5 A7 A8 A6 A6 A7
A8 A8 A7 A7 A10 A10 A8 A9 A9
..................

O2 layout:

2 drives 3 drives 4 drives
-------------------------------------
A1 A2 A1 A2 A3 A1 A2 A3 A4
A2 A1 A3 A1 A2 A4 A1 A2 A3
A3 A4 A4 A5 A6 A5 A6 A7 A8
A4 A3 A6 A4 A5 A5 A6 A7
A5 A6 A7 A8 A9 A10 A11 A12
A6 A5 A9 A7 A8 A12 A9 A10 A11
..................

[3] key parameters in the kernel

Drivers/MD/raid10.h

Struct r10_private_data_s {
......
/* Geometry */
Int near_copies;/* Number of copies layed out raid0 style */
Int far_copies;/* Number of copies layed out
* At large strides implements SS drives
*/
Int far_offset;/* far_copies are offset by 1 stripe
* Instead of bytes
*/
Int copies;/* near_copies * far_copies.
* Must Be <= raid_disks
*/
Sector_t stride;/* distance between far copies.
* This is size/far_copies unless
* Far_offset, in which case it is
* 1 stripe.
*/
......
};

Near_copies: each chunk has a copy of near_copies (including its own)

Far_copies: Each device is divided into far_copies sections.

Far_offset: If the offset value is true, the offset mode is started, and the same Chunk is located at different offsets of adjacent devices.

References:

[1] http://neil.brown.name/blog/20040827225440

[2] http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10

[3] MAN mdadm

[4] MAN MD

[5] Linux-2.6.18kernel source code