FAQs about Linux RAID

In the article "use of software RAID in Linux", we talked about the basic concepts of software RAID in Linux and how to use mdadm program to manage software RAID devices, this section describes how to create, delete, stop, and expand a software RAID device, and how to manage its metadata and bitmap data. In this article, we will discuss common problems and solutions for using soft RAID devices.
1. The RAID sub-devices must be physically independent and unrelated.

RAID on multiple disk partitions on a disk is of no value, because not only does it violate the principle of device independence in the RAID array, but it cannot increase the reliability of the array, when the disk fails, data loss is inevitable, and the performance of the array is significantly reduced, when data is read/written, the head jitters back and forth on multiple partitions of the disk, resulting in a long search time.

2. RAID 0 has no data redundancy Function

RAID 0 can only use the data block-based (stripping) function to improve performance. If a disk fails, data on the MD device is lost and an error occurs when reading and writing the MD device.

3. Quick test command

When creating a disk, if the data on the device is already synchronized (sync), or only used to test the array, you can use the -- assume-clean parameter to notify the MD driver that the array does not need to be initialized.

Mdadm-C/dev/md0-l5-n6/dev/sd [B-g]-x1/dev/sdh -- assume-clean

If you want to avoid the previous metadata prompts on the device during each command creation by mdadm, and avoid manual input, you can use -- run (or its abbreviation is-R) to notify the array to run the command directly.

Mdadm -- create -- run/dev/md0-l5-n6/dev/sd [B-g]-x1/dev/sdh -- assume-clean

4. RAID composed of heterogeneous Disks

RAID0 supports the construction of RAID in multiple zones on disks of different sizes. Each region has a different number of disks. Use the fdisk command to query whether/dev/sdi1 is 2 GB,/dev/sdj1 is 4 GB, And/dev/sdk1 is 1 GB. Therefore, the RAID 0 device is 7 GB.

Mdadm-C/dev/md0-l5-n6/dev/sd [B-g]-x1/dev/sdh -- assume-clean

For disks of different sizes such as RAID1, RAID456, and RAID10, only the minimum disk capacity can be used as the public size, and the excess size is wasted. The RAID 5 device made of/dev/sdi1,/dev/sdj1,/dev/sdk1 is 2 GB, which is twice that of the smallest device/dev/sdk1, the device/dev/sdi1 and sdj1 waste 1 GB and 3 GB respectively.

[Root @ fc5 mdadm-2.6.3] #. /mdadm-CR/dev/md1-l0-n3/dev/sd [I-k] 1 [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md1 | grep "Array Size" Array Size: 7337664 (7.00 GiB 7.51 GB)

The size (ArraySize) in the array status information query is KB, indicating the actual size of the MD block device. The following two values are only used to display the processed results.

[Root @ fc5 mdadm-2.6.3] #. /mdadm-CR/dev/md1-l0-n3/dev/sd [I-k] 1 [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md1 | grep "Array Size" Array Size: 7337664 (7.00 GiB 7.51 GB) [root @ fc5 mdadm-2.6.3] #. /mdadm-CR/dev/md1-l5-n3/dev/sd [I-k] 1 [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md1 | grep "Array Size" Array Size: 2096896 (2048.09 MiB 2147.22 MB)

5. Configure the shared hot spare Disk

The mdadm program allows multiple RAID groups to share redundant disks. For example, there are two Arrays:/dev/md0 and/dev/md1. At the time of creation,/dev/md0 contains a hot backup disk, while/dev/md1 does not have a hot backup disk. We only need to configure two arrays in/etc/mdadm. conf to use the same spare-group.

[Root @ fc5 mdadm-2.6.3] # cat/etc/mdadm. confDEVICE/dev/sdb/dev/sdc/dev/sdd/dev/sde/dev/sdf/dev/sdg/dev/sdh/dev/sdi1/dev/sdj1/dev/ sdk1 ARRAY/dev/md1 level = raid0 num-devices = 3 spare-group = sparedisks UUID = dcff6ec9: 53c4c668: 58b81af9: ef71989d ARRAY/dev/md0 level = raid 10 num-devices = 6 spare-group = sparedisks UUID = 0cabc5e5: 842d4baa: e3f6261b: a17a477a

And run the mdadm monitoring mode command. When a disk/dev/sdi1 in the/dev/md1 array fails, mdadm automatically removes the spare disk from the/dev/md0 group and adds it to/dev/md1.

[Root @ fc5 mdadm-2.6.3] #. /mdadm -- monitor -- mail = root @ localhost -- syslog -- program =/root/md. sh -- delay = 300/dev/md * -- daemonise 8105 [root @ fc5 mdadm-2.6.3] #. /mdadm/dev/md1-f/dev/sdi1mdadm: set/dev/sdi1 faulty in/dev/md1 [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md1/dev/md1: Version: 00.90.03 Creation Time: Mon Aug 23 00:10:00 1999 Raid Level: raid5 Array Size: 2096896 (2048.09 MiB 2147.22 MB) used Dev Size: 1048448 (1024.05 MiB 1073.61 MB) Raid Devices: 3 Total Devices: 4 Preferred Minor: 1 Persistence: Superblock is persistent Update Time: Mon Aug 23 00:13:15 1999 State: clean, degraded, recovering Active Devices: 2 Working Devices: 3 Failed Devices: 1 Spare Devices: 1 Layout: left-blank Ric Chunk Size: 64 K Rebuild Status: 78% complete UUID: 34d3de75: 6912dc24: e1316607: 4e72cd01 Events: 0.4 Number Major Minor RaidDevice State 3 8 112 0 spare rebuilding/dev/sdh 1 8 145 1 active sync/dev/sdj1 2 8 161 2 active sync/dev/sdk1 4 8 129- faulty spare/dev/sdi1

6. Multiple metadata formats

The MD device has two main metadata formats (version 0.9 and 1. version x), mainly because the space occupied by metadata is different from the number of underlying devices supported, and 1. version x can also specify the location where the metadata is written to the device (1.0 is at the end of the device, 1.1 is at the start of the device, and 1.2 is at the 4KB position at the beginning of the device ). You can use the -- metadata (or its abbreviation-e) parameter to create an array to specify the metadata format version.

[Root @ fc5 mdadm-2.6.3] #. /mdadm-CR/dev/md0-l5-n6-x1/dev/sd [B-h] [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md0 | grep Version: 00.90.03 [root @ fc5 mdadm-2.6.3] #. /mdadm-CR/dev/md0-l5-n6-x1-e1.0/dev/sd [B-h] [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md0 | grep Version: 01.00.03 [root @ fc5 mdadm-2.6.3] #. /mdadm-CR/dev/md0-l5-n6-x1-e1.1/dev/sd [B-h] [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md0 | grep Version: 01.01.03 [root @ fc5 mdadm-2.6.3] #. /mdadm-CR/dev/md0-l5-n6-x1-e1.2/dev/sd [B-h] [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md0 | grep Version: 01.02.03

When the array removes the disk, it also uses -- zero-superblock to clear the metadata information on the device.

[Root @ fc5 mdadm-2.6.3] #. /mdadm-Es/dev/sdhARRAY/dev/md0 level = raid5 num-devices = 6 UUID = acd2b182: 4695ee20: 37689502: eb0423e1 spares = 1 [root @ fc5 mdadm-2.6.3] #. /mdadm/dev/md0-f/dev/sdh-r/dev/sdhmdadm: set/dev/sdh faulty in/dev/md0mdadm: hot removed/dev/sdh [root @ fc5 mdadm-2.6.3] #. /mdadm -- zero-superblock/dev/sdh [root @ fc5 mdadm-2.6.3] #. /mdadm-Es/dev/sdh [root @ fc5 mdadm-2.6.3] #

7. partitioned RAID Devices

If you want to partition the MD devices that support partitions (Partitionable raid array), you must use/dev/md_d0 to replace the previous/dev/md0 when creating the device. The -- auto = mdp (or-ap) parameter is used to create an array.

[Root @ fc5 mdadm-2.6.3] #. /mdadm-Cv -- auto = mdp/dev/md_d0-l5-n6/dev/sd [B-g]-x1/dev/sdh mdadm: layout defaults to left-direction ricmdadm: chunk size defaults to 64 Kmdadm: size set to 1048512 Kmdadm: array/dev/md_d0 started. [root @ fc5 mdadm-2.6.3] # fdisk/dev/md_d0Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel. changes will remain in me Mory only, until you decide to write them. after that, of course, the previuscontent won't be recoverable. the number of cylinders for this disk is set to 1310640. there is nothing wrong with that, but this is larger than 1024, and cocould in certain setups cause problems with: 1) software that runs at boot time (e.g ., old versions of LILO) 2) booting and partitioning software from other OSs (e.g ., DOS F DISK, OS/2 FDISK) Warning: invalid flag 0x0000 of partition table 4 will be corrected by w (rite) Command (m for help ): nCommand action e extended p primary partition (1-4) pPartition number (1-4): 1 First cylinder (1-1310640, default 1 ): 1 Using default value 1 Last cylinder or + size or + sizeM or + sizeK (1-1310640, default 1310640): + 1024 MCommand (m for help ): nCommand action e extended p primary partiti On (1-4) pPartition number (1-4): 2 First cylinder (250002-1310640, default 250002 ): 250002 Using default value 250002 Last cylinder or + size or + sizeM or + sizeK (250002-1310640, default 1310640): 1310640 Using default value 1310640 Command (m for help ): w The partition table has been altered! Calling ioctl () to re-read partition table. syncing disks. [root @ fc5 mdadm-2.6.3] # fdisk-l/dev/md_d0Disk/dev/md_d0: 5368 MB, 5368381440 bytes2 heads, 4 sectors/track, 1310640 cylindersUnits = cylinders of 8*512 = 4096 bytes Device Boot Start End Blocks Id System/dev/md_d0p1 1 250001 1000002 83 Linux/dev/md_d0p2 250002 1310640 4242556 83 Linux

The MD device supports two types of devices: The unpartitioned type is the md device, the master device number is 9, and the partition name is the mdp device, the mdp master device number is dynamically allocated, and a maximum of 63 partitions are supported for one mdp device. View the/proc/device information to see the mdp master device number. query the/proc/partitions information to see the md and mdp master device numbers and slave device numbers.

[Root @ fc5 mdadm-2.6.3] # cat/proc/devices | grep md1 ramdisk9 md253 mdp [root @ fc5 mdadm-2.6.3] # cat/proc/partitions | grep md 9 1 2096896 md1 253 0 5242560 md_d0 253 1 1000002 md_d0p1 253 2 1000002 md_d0p2

8. How to expand RAID Devices

The capacity expansion function in Grow mode not only supports increasing the device size, but also reduces the device size. However, the user is required to ensure that the effective data of the MD device is not truncated, resulting in loss.

[Root @ fc5 mdadm-2.6.3] #. /mdadm/dev/md0 -- grow -- size = 102400 [root @ fc5 mdadm-2.6.3] #. /mdadm-Q/dev/md0/dev/md0: 600.00MiB raid5 7 devices, 0 spares. use mdadm -- detail for more detail.

The reshape function reduces the number of devices that are not supported. If the backup file is not specified for an idle hot backup disk in the array, it cannot be expanded. If a backup file is available for the RAID5 array, you can expand a disk. The extended disk array is in the downgrade mode, you cannot expand multiple disks or downgrade the RAID 5 array. These extended modes are not safe enough. They only do not have the right to use hot spare disks for the moment. Generally, it is reasonable to add idle hot backup disks before executing the extended commands.

[Root @ fc5 mdadm-2.6.3] #. /mdadm -- grow/dev/md0-n6mdadm:/dev/md0: Cannot reduce number of data disks (yet ). [root @ fc5 mdadm-2.6.3] #. /mdadm -- grow/dev/md0-n8mdadm: Need to backup 2688 K of critical section. mdadm:/dev/md0: Cannot grow-need a spare or backup-file to backup critical section [root @ fc5 mdadm-2.6.3] #. /mdadm-G/dev/md0-plugin -- backup-file =/root/bak-reshapemdadm: Need to backup 1024 K of critical section .. mdadm:/dev/md0: failed to initiate reshape

[Root @ fc5 mdadm-2.6.3] #. /mdadm-G/dev/md0-n8 -- backup-file =/root/bak-reshapemdadm: Need to backup 2688 K of critical section .. mdadm :... critical section passed. [root @ fc5 mdadm-2.6.3] # cat/proc/mdstatPersonalities: [raid0] [raid10] [raid6] [raid5] [raid4] md0: active raid5 sdh [6] sdg [5] sdf [4] sde [3] sdd [2] sdc [1] sdb [0] 6291072 blocks super 0.91 level 5, 64 k chunk, algorithm 2 [8/7] [UUUUUUU _] [> ......] reshape = 2.6% (28608/1048512) finish = 4.1 min speed = 4086 K/secmd1: active raid0 sdk1 [2] sdj1 [1] sdi1 [0] 7337664 blocks 64 k chunksunused devices: <none> [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md0/dev/md0: Version: 00.91.03 Creation Time: Sun Aug 22 23:46:29 1999 Raid Level: raid5 Array Size: 6291072 (6.00 GiB 6.44 GB) used Dev Size: 1048512 (1024.11 MiB 1073.68 MB) Raid Devices: 8 Total Devices: 7 Preferred Minor: 0 Persistence: Superblock is persistent Update Time: Sun Aug 22 23:46:56 1999 State: clean, degraded, recovering Active Devices: 7 Working Devices: 7 Failed Devices: 0 Spare Devices: 0 Layout: left-blank Ric Chunk Size: 64 K Reshape Status: 4% complete Delta Devices: 1, (7-> 8) UUID: 50e886b9: 9e28d243: a7f6457d: 5eb47f2e Events: 0.94 Number Major Minor RaidDevice State 0 8 16 0 active sync/dev/sdb 1 8 32 1 active sync/dev/sdc 2 8 48 2 active sync/dev/sdd 3 8 64 3 active sync/dev/sde 4 8 80 4 active sync/dev/sdf 5 8 96 5 active sync/dev/sdg 6 8 112 6 active sync/dev/sdh 7 0 0 7 removed [root @ fc5 mdadm-2.6.3] # cat/proc/mdstatPersonalities: [raid0] [raid10] [raid6] [raid5] [raid4] md0: active raid5 sdh [6] sdg [5] sdf [4] sde [3] sdd [2] sdc [1] sdb [0] 7339584 blocks level 5, 64 k chunk, algorithm 2 [8/7] [UUUUUUU _] md1: active raid0 sdk1 [2] sdj1 [1] sdi1 [0] 7337664 blocks 64 k chunksunused devices: <none> [root @ fc5 mdadm-2.6.3] #. /mdadm-D/dev/md0/dev/md0: Version: 00.90.03 Creation Time: Sun Aug 22 23:46:29 1999 Raid Level: raid5 Array Size: 7339584 (7.00 GiB 7.52 GB) used Dev Size: 1048512 (1024.11 MiB 1073.68 MB) Raid Devices: 8 Total Devices: 7 Preferred Minor: 0 Persistence: Superblock is persistent Update Time: Sun Aug 22 23:50:18 1999 State: clean, degraded Active Devices: 7 Working Devices: 7 Failed Devices: 0 Spare Devices: 0 Layout: left-encrypted Ric Chunk Size: 64 k uuid: Fingerprint: 9e28d243: a7f6457d: 5eb47f2e Events: 0.766 Number Major Minor RaidDevice State 0 8 16 0 active sync/dev/sdb 1 8 32 1 active sync/dev/sdc 2 8 48 2 active sync/dev/sdd 3 8 64 3 active sync/dev/sde 4 8 80 4 active sync/dev/sdf 5 8 96 5 active sync/dev/sdg 6 8 112 6 active sync/dev/sdh 7 0 0 7 removed [root @ fc5 mdadm-2.6.3] #. /mdadm-G/dev/md0-plugin -- backup-file =/root/bak-reshapemdadm: Need to backup 3584 K of critical section .. mdadm:/dev/md0: failed to find device 7. array might be degraded. -- grow aborted

9. Why cannot I extend the RAID device?

If the grow mode is used, an error is returned if the kernel does not support sysfs or the MD driver version is not supported. In addition, the strace command is used to track the function calling errors that can be returned.

[Root @ fc5 mdadm-2.6.3] #. /mdadm -- grow/dev/md0-n6mdadm: Need to backup 1280 K of critical section .. mdadm:/dev/md0: Cannot get array details from sysfs

If the system does not load sysfs, use the mount command to load it.

Mount-t sysfs/sys/

10. Why cannot I stop the MD device?

When the MD array is stopped, it is sometimes returned that the MD device is being used incorrectly. You need to check which upper-layer device is using the MD device. One scenario is used by the file system, such as being mounted:

[Root @ fc5 mdadm-2.6.3] # df-h | grep md/dev/md0 485 M 11 M 449 M 3%/mnt/md-test

Another scenario is that the MD device is opened and used by other programs on the user layer, such as nbd-server exporting the MD device:

[Root @ fc5 md0] # fuser-u/dev/md0/dev/md0: 3507 (root) [root @ fc5 md0] # ps ax | grep 3507 3507?

It is also possible that the kernel driver is enabled and used as a physical volume (PV) by LVM programs:

[Root @ fc5 mdadm-2.6.3] # ls-l/sys/block/md0/holders/total 0 lrwxrwxrwx 1 root 0 08-22 dm-0-> .. /.. /.. /block/dm-0

Summary

If the problem cannot be solved during use of soft RAID, you should first check the mdadm help information. If the problem persists, you can go to the mdadm mail list to find out if you have any related questions or describe the problem occurrence process in detail.

References

Learning

The man mdadm command is used to view the help information of the mdadm program. Each parameter is explained and a simple example is provided.

The test set of the mdadm command in the tests directory of the mdadm source program provides typical commands and parameters, and detailed usage of various parameters of the mdadm command.

Obtain products and technologies

Download mdadm is a software RAID user layer management software in Linux.

Discussion

The mdadm Mail List linux-raid@vger.kernel.org discusses a lot of soft RAID usage tips and problem solutions, if you encounter problems in the use of soft RAID, you can go to the above to find the relevant problem or request help.

About the author

Zhang zhibin, a fan of Linux systems and open-source software, is welcome to share their experience in using open-source software.