Software RAID configuration in CentOS

Software RAID configuration in CentOS

This article will explain in detail the basic principles of RAID and the configuration of software RAID in Linux.

I. Basic principles of RAID

Before explaining the principles of RAID, Let's first look at the disadvantages of traditional disks. We know that a PC can contain CPU, memory, motherboard, hard disk, Nic, and other hardware. The components that affect computer performance include: CPU, motherboard bus IO, memory IO, hard disk IO, Nic IO, etc. Maybe when we mention the impact on computer performance, we first think of CPU. However, with the development of computers, especially for modern processors, the computation speed is already very fast, at the same time, our memory I/O speed has reached a very fast point (almost 5 GB per second), and we also know that data is stored on the hard disk, therefore, the computer transfers the hard disk data to the memory, and then the CPU loads the data from the memory for calculation, so it seems that the factor affecting the performance of the entire computer is our hard disk I/O speed. Let's take a look at the current popular hard drive types and speeds (data may be inaccurate, but it is almost the same)

　　Hard Disk TypeSpeed SATA <150 M/s SCSI <200 M/s SAS200M/s SSD around 500 MB/S

Our current PCs are basically hard disks using SATA interfaces. The read speed cannot exceed 150 Mb/s, And the write speed is slower, services in the production environment basically use SAS (Serial SCSI) hard disks. The fastest speed is ssd solid state disks, which are almost 4-5 times faster than SATA. However, even if SSD is used, its speed is around 500 Mb/s, which is far below our memory and CPU processing speed. So,HardDisks are the performance bottleneck of most computers.

Therefore, the defect of modern disks is:I/O performance is very poor, and stability is very poor.

I/O performance we have just seen that, even if SSD is used, it will greatly affect the performance of the computer. Poor stability is manifested in the fact that if a hard disk is faulty or damaged, this hard disk can no longer be used, which is unimaginable if it is in a place with extremely high requirements for data storage. As a result, RAID is a new technology.

RAID (Redundant Array of Independent Disks) is short for the Redundant Array Technology of cheap Disks. Its principle isI/O storage performance of the entire computer is improved by running multiple disks in parallel.

RAID has three criteria:

①Speed: read/write speed improvement

②Disk usage: space usage of multiple disks

③Redundancy: several disks can be damaged without data loss.

Therefore, based on the above three criteria, RAID is divided into many types, called RAID levels. A total of seven levels of modern RAID are available: RAID 0 ~ RAID 6, but the commonly used RAID level is mainly the following four types:

①RAID0: Improves read/write Performance

②RAID1: Improves read/write performance and Redundancy

③RAID5: Improves read/write performance and redundancy (allows failure of one hard disk)

④RAID6: Improves read/write performance and redundancy (two hard disks are faulty)

Next we will look at the features of the four RAID levels based on the three criteria of RAID.

1. RAID0

The principle of RAID is to improve the I/O storage performance of the entire computer by running multiple hard disks in parallel. Therefore, for the level of RAID0, we need at least two hard disks. When reading and writing data, RAID0 improves Read and Write Performance by separately reading and writing data to multiple hard disks. Let's take a look at how RAID0 works.

RAID0 requires at least two hard disks. When RAID0 is used, we read and write data separately to multiple hard disks, so the speed of reading and writing is the fastest, however, because a part of data is stored on multiple hard disks, the entire RAID data is damaged when a hard disk is damaged.

① Space utilization: sum of all hard disk space

② Performance: Sum of the read/write speeds of all hard disks

③ Redundancy: None

2. RAID1

RAID1 also requires at least two hard disks. When writing data, it is different from RAID0. When writing data, RAID1 will copy the data to multiple hard disks, that is, each Hard Disk stores a backup of the data to improve redundancy when reading data. Read data from multiple hard disks at the same time to improve reading performance.

① Space utilization: The smallest disk (in fact, the size and model of each hard disk are the same when RAID is used)

② Performance: the read performance is the sum of all hard disks, and the write performance is weakened.

③ Redundancy: data is normal as long as one hard disk is normal.

3. RAID5

RAID5 requires at least three hard disks. RAID5 is similar to RAID0. When reading and writing data, the data is distributed to all hard disks. However, when writing data, RAID5 performs a parity operation on the data and stores the verification information on the hard disk. Even if one of our hard disks is damaged, RAID5 can also use other hard disks and verification information to restore data. However, if two or more hard disks are damaged, the entire data is damaged.

① Space utilization: 1-1/n

② Performance: the read performance is close to RAID0, And the write performance is weaker than RAID0.

③ Redundancy: one hard disk can be damaged.

4. RAID6

RAID6 requires at least four hard disks. RAID6 is similar to RAID5. When reading and writing data, the data is distributed to all hard disks. When writing data, RAID5 performs a parity operation on the data and stores the verification information on the hard disk. However, RAID6 saves more verification information than RAID5, therefore, the redundancy of RAID 6 is higher than that of RAID 5. Two hard disks can be damaged.

① Space utilization: 1-2/n

② Performance: the read performance is close to RAID5, And the write performance is weaker than RAID5.

③ Redundancy: Damage to two hard disks is acceptable.

The above four RAID levels are the four most commonly used levels. For personal PCs, what we need most to improve is the hard disk storage performance, so we basically use RAID0, its read/write performance has been greatly improved, but its redundancy is 0. When the hard disk is damaged, the data is damaged. In the production environment, the server uses RAID 5 or raid 6 at most, which provides read/write performance and redundancy. RAID1 is usually used only when there are strict requirements on data accuracy.

We will summarize the advantages and disadvantages of the four commonly used RAID levels:

　　RAID levelSpeedRedundancyDisk utilization RAID 0 read/write speed increase 0 sum of all disks RAID 1 read Speed prompt n one disk size RAID 5 read/write speed increase 1-1/n RAID 6 read/write speed increase 2 1-2/n

RAID can be implemented in two ways:Software RAID and hardware RAID

① Software RAID

RAID can be implemented through system functions or RAID software. Without independent hardware and interfaces, the system resources (CPU and hard disk interface speed) need to be occupied and the operating system stability is affected.

② Hardware RAID

Using an independent RAID hardware card, some boards are integrated with RAID hardware, some need to purchase an independent RAID hardware card, hardware RAID implementation does not need to occupy other hardware resources, and the stability and speed are better than software RAID, therefore, for servers, it is best to use hardware RAID to improve computer performance.

Ii. Use of software RAID in Linux

Currently, all operating systems, including windows, mac OS, and linux, have software RAID implementation.Implemented by the mdadm Program

UsePrecautions for the mdadm software:

① Mdadm supports RAID levels: RAID0, RAID1, RAID4, RAID5, and RAID6. We can see that mdadm supports the four commonly used RAID levels.

② Mdadm can create RAID based on multiple hard disks, partitions, and logical volumes. For hardware RAID, it can only be based on multiple hard disks.

③ The created software RAID corresponds to/dev/mdn, and n represents the first RAID, for example, the first RAID corresponds to/dev/md0, the second RAID corresponds to/dev/md1. Of course, this name can be obtained by yourself.

④ The RAID information is stored in the/proc/mdstat file or viewed using the mdadm command.

Next, I will create our software RAID on my CentOS system.

Before creating software RAID, I first simulated four 1g virtual hard disks through the Virtual Machine. Of course, in the actual environment, the specific hard disk is used.

[root@xiaoluo ~]# fdisk -lDisk /dev/sda: 21.5 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00093d90 Device Boot Start End Blocks Id System/dev/sda1 1 523 4194304 82 Linux swap / SolarisPartition 1 does not end on cylinder boundary./dev/sda2 * 523 2611 16776192 83 LinuxDisk /dev/sdb: 1073 MB, 1073741824 bytes255 heads, 63 sectors/track, 130 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdc: 1073 MB, 1073741824 bytes255 heads, 63 sectors/track, 130 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdd: 1073 MB, 1073741824 bytes255 heads, 63 sectors/track, 130 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sde: 1073 MB, 1073741824 bytes255 heads, 63 sectors/track, 130 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000

We created the software RAID through the mdadm command. For example, we created a RAID 0. The syntax format is as follows:

Create RAID 0: mdadm-C/dev/md0-a yes-l 0-n 2/dev/sdb/dev sdc

①-C: Create a new RAID. Here we create the first RAID named/dev/md0.

②-A automatically creates the corresponding device. yes indicates that the RAID device is automatically created under/dev.

③-L specify the RAID level to be created. 0 is created here.

④-N indicates the number of hard disks. 2 indicates that the RAID 0 is created using two hard disks:/dev/sdb and/dev/sdc.

We use the mdadm command to create the syntax format of the software RAID.

Create RAID 1: mdadm-C/dev/md1-a yes-l 1-n 2/dev/sdb/dev/sdc create RAID 5: mdadm-C/dev/md2-a yes-l 5-n 3/dev/sdb/dev/sdc/dev/sdd create RAID 6: mdadm-C/dev/md3-a yes-l 6-n 4/dev/sdb/dev/sdc/dev/sdd/dev/sde

We can also use the-x parameter to specify a backup disk, which is generally not used. When a disk failure occurs, the specified backup disk can be automatically launched:

mdadm -C /dev/md0 -a yes -l 5 -n 3 -x /dev/sdb /dev/sdc /dev/sdd /dev/sde

We can see that we have created a RAID 5 and used three hard disks. At this time, we have specified the parameter-x, indicating that we have specified a hard disk as a backup disk, when one of the other three disks fails, the specified backup hard disk can be automatically launched.

Next we will create a RAID 0 to see how to use the mdadm command:

[root@xiaoluo ~]# mdadm -C /dev/md0 -a yes -l 0 -n 2 /dev/sdb /dev/sdc mdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md0 started.[root@xiaoluo ~]# ls -l /dev/md0 brw-rw----. 1 root disk 9, 0 May 25 22:36 /dev/md0

Now we have created a RAID 0, and we found that a device named md0 exists under/dev, you can use the mdadm-D command to view the details of the RAID you just created, or view the/proc/mdstat file to view RAID information.

[Root @ xiaoluo ~] # Cat/proc/mdstat Personalities: [raid0] md0: active raid0 sdc [1] sdb [0] 2096128 blocks super 1.2 512 k chunks unused devices: <none> [root @ xiaoluo ~] # Mdadm-Dmdadm: No devices given. [root @ xiaoluo ~] # Mdadm-D/dev/md0/dev/md0: Version: 1.2 Creation Time: Sat May 25 22:36:15 2013 Raid Level: raid0 Array Size: 2096128 (2047.34 MiB 2146.44 MB) raid Devices: 2 Total Devices: 2 Persistence: Superblock is persistent Update Time: Sat May 25 22:36:15 2013 State: clean Active Devices: 2 Working Devices: 2 Failed Devices: 0 Spare Devices: 0 Chunk Size: 512 K Name: xiaoluo: 0 (local to host xiaoluo) UUID: fe746431: 4d77f0e9: e1c1a06f: 1d341790 Events: 0 Number Major Minor RaidDevice State 0 8 16 0 active sync/dev/sdb // created RAID 0 use the two hard disks/dev/sdb/dev/sdc 1 8 32 1 active sync/dev/sdc

【Note: after creating a RAID, we need to save the RAID information to/etc/mdadm. in the conf file, the system will automatically load the file to enable RAID when the next operating system restarts.

[root@xiaoluo ~]# mdadm -D --scan > /etc/mdadm.conf[root@xiaoluo ~]# cat /etc/mdadm.conf ARRAY /dev/md0 metadata=1.2 name=xiaoluo:0 UUID=fe746431:4d77f0e9:e1c1a06f:1d341790

In this way, RAID will be automatically enabled when the system restarts next time.

After this RAID 0 is created, we cannot use the/dev/sdb and/dev/sdc hard disks any more, it will damage the RAID we just created, so we use RAID 0 to format and mount the file system.

[Root @ xiaoluo ~] # Mkfs. ext4/dev/md0 mke2fs 1.41.12 (17-May-2010) Filesystem label = OS type: LinuxBlock size = 4096 (log = 2) Fragment size = 4096 (log = 2) stride = 128 blocks, Stripe width = 256 blocks131072 inodes, 524032 blocks26201 blocks (5.00%) reserved for the super userFirst data block = 0 Maximum filesystem blocks = 53687091216 block groups32768 blocks per group, 32768 fragments per group8192 inodes per groupSuperblock backu Ps stored on blocks: 32768,983 04, 163840,229 376, 294912 Writing inode tables: done Creating journal (8192 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 24 mounts or180 days, whichever comes first. use tune2fs-c or-I to override. [root @ xiaoluo ~] # Mount/dev/md0/mnt [root @ xiaoluo ~] # Mount/dev/sda2 on/type ext4 (rw) proc on/proc type proc (rw) sysfs on/sys type sysfs (rw) devpts on/dev/pts type devpts (rw, gid = 5, mode = 620) tmpfs on/dev/shm type tmpfs (rw, rootcontext = "system_u: object_r: tmpfs_t: s0 ") none on/proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on/var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/md0 on/mnt type ext4 (rw) // Our/dev/md0 has been mounted and can be used [root @ xiaoluo ~] # Cd/mnt/[root @ xiaoluo mnt] # lslost + found

In this case, we use RAID to perform file operations. The RAID level is used during creation, so the RAID will have read/write features at this level. After creating a RAID, we can use the RAID as if using a partition.

We can also use the mdadm-S command to disable RAID. Of course, before stopping RAID, We need to detach the RAID first.

[Root @ xiaoluo mnt] # cd
[Root @ xiaoluo ~] # Umount/mnt

[Root @ xiaoluo ~] # Mdadm-S/dev/md0
Mdadm: stopped/dev/md0

Generally, if we want to restart our RAID, we can use the mdadm-R command, but it may be because of the operating system or software version. After the RAID is disabled, using this command will prompt that the file cannot be found

[root@xiaoluo ~]# mdadm -R /dev/md0mdadm: error opening /dev/md0: No such file or directory

At this time, we only need to restart the operating system, because we have saved the RAID information in the/etc/mdadm. conf file.

[root@xiaoluo ~]# ls -l /dev/md0brw-rw----. 1 root disk 9, 0 May 25 22:57 /dev/md0

We can see that after the operating system is restarted, The RAID device had another one.

If I want to remove a hard disk from the device at this time, or I want to completely delete the RAID so that we can use the hard disk we just used as RAID as before, what should we do at this time?

Run the mdadm -- zero-superblock command, but we must first stop our RAID by using the mdadm-S command. For example, to completely remove the two hard disks created in RAID 0, you can run the following command:

[root@xiaoluo ~]# mdadm -S /dev/md0mdadm: stopped /dev/md0[root@xiaoluo ~]# mdadm --zero-superblock /dev/sdb [root@xiaoluo ~]# mdadm --zero-superblock /dev/sdc

At this time, all the RAID 0 information is cleared, and we can use the two hard disks separately.

Here we will try to create a RAID 5, and then explain how to simulate a fault command.

[Root @ xiaoluo ~] # Mdadm-C/dev/md0-a yes-l 5-n 3/dev/sdb/dev/sdc/dev/sdd mdadm: Defaulting to version 1.2 metadatamdadm: array/dev/md0 started. [root @ xiaoluo ~] # Cat/proc/mdstat Personalities: [raid0] [raid6] [raid5] [raid4] md0: active raid5 sdd [3] sdc [1] sdb [0] 2095104 blocks super 1.2 level 5,512 k chunk, algorithm 2 [3/2] [UU _] [=======================>...] recovery = 87.2% (915204/1047552) finish = 0.0 min speed = 91520 K/sec // The operation is not completed unused devices: <none>

【Note: When we create RAID 5 or RAID 6, we need to check the hard disk because it may vary according to the size and time of the hard disk, after entering the mdadm command, we must also check the/proc/mdstat file to check whether the progress information in the file is complete. For example, the above creation time is only 87.2%, the following operations can be performed only after the progress is displayed completely.

[Root @ xiaoluo ~] # Cat/proc/mdstat Personalities: [raid0] [raid6] [raid5] [raid4] md0: active raid5 sdd [3] sdc [1] sdb [0] 2095104 blocks super 1.2 level 5,512 k chunk, algorithm 2 [3/3] [UUU] // after the operation is completed, unused devices: <none>

You can use mdadm-D to view RAID details:

[root@xiaoluo ~]# mdadm -D /dev/md0/dev/md0: Version : 1.2 Creation Time : Sat May 25 23:07:06 2013 Raid Level : raid5 Array Size : 2095104 (2046.34 MiB 2145.39 MB) Used Dev Size : 1047552 (1023.17 MiB 1072.69 MB) Raid Devices : 3 Total Devices : 3 Persistence : Superblock is persistent Update Time : Sat May 25 23:07:18 2013 State : clean Active Devices : 3Working Devices : 3 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 512K Name : xiaoluo:0 (local to host xiaoluo) UUID : 029e2fe7:8c9ded40:f5079536:d249ccf7 Events : 18 Number Major Minor RaidDevice State 0 8 16 0 active sync /dev/sdb 1 8 32 1 active sync /dev/sdc 3 8 48 2 active sync /dev/sdd

In the experimental environment, we can also use the mdadm command to simulate RAID faults and run the mdadm/dev/md0-f/dev/sdd command.

[root@xiaoluo ~]# mdadm /dev/md0 -f /dev/sddmdadm: set /dev/sdd faulty in /dev/md0

We can check the RAID information again:

[Root @ xiaoluo ~] # Mdadm-D/dev/md0/dev/md0: Version: 1.2 Creation Time: Sat May 25 23:07:06 2013 Raid Level: raid5 Array Size: 2095104 (2046.34 MiB 2145.39 MB) used Dev Size: 1047552 (1023.17 MiB 1072.69 MB) Raid Devices: 3 Total Devices: 3 Persistence: Superblock is persistent Update Time: Sat May 25 23:13:44 2013 State: clean, degraded Active Devices: 2 Working Devices: 2 Failed Devices: 1 Spare Devices: 0 Layout: left-blank Ric Chunk Size: 512 K Name: xiaoluo: 0 (local to host xiaoluo) UUID: 029e2fe7: 8c9ded40: f5079536: d249ccf7 Events: 19 Number Major Minor RaidDevice State 0 8 16 0 active sync/dev/sdb 1 8 32 1 active sync/dev/sdc 2 0 0 2 removed 3 8 48-faulty spare/dev/ sdd // This hard disk is marked as a bad Hard Disk

We can see that the hard disk/dev/sdd is marked as a bad hard disk, because we are using the RAID 5 level, so a hard disk is damaged and will not cause data damage, the data is still intact.

We can remove this hard disk through mdadm/dev/md0-r/dev/sdd.

[Root @ xiaoluo ~] # Mdadm/dev/md0-r/dev/sdd mdadm: hot removed/dev/sdd from/dev/md0 [root @ xiaoluo ~] # Mdadm-D/dev/md0/dev/md0: Version: 1.2 Creation Time: Sat May 25 23:07:06 2013 Raid Level: raid5 Array Size: 2095104 (2046.34 MiB 2145.39 MB) used Dev Size: 1047552 (1023.17 MiB 1072.69 MB) Raid Devices: 3 Total Devices: 2 Persistence: Superblock is persistent Update Time: Sat May 25 23:17:12 2013 State: clean, degraded Active Devices: 2 Working Devices: 2 Failed Devices: 0 Spare Devices: 0 Layout: left-blank Ric Chunk Size: 512 K Name: xiaoluo: 0 (local to host xiaoluo) UUID: 029e2fe7: 8c9ded40: f5079536: d249ccf7 Events: 22 Number Major Minor RaidDevice State 0 8 16 0 active sync/dev/sdb 1 8 32 1 active sync/dev/sdc 2 0 0 2 removed // dev/sdd has been removed dropped

If you want to replace it with a new hard disk, you can use the mdadm/dev/md0-a/dev/sde command.

[Root @ xiaoluo ~] # Mdadm/dev/md0-a/dev/sde mdadm: added/dev/sde [root @ xiaoluo ~] # Mdadm-D/dev/md0/dev/md0: Version: 1.2 Creation Time: Sat May 25 23:07:06 2013 Raid Level: raid5 Array Size: 2095104 (2046.34 MiB 2145.39 MB) used Dev Size: 1047552 (1023.17 MiB 1072.69 MB) Raid Devices: 3 Total Devices: 3 Persistence: Superblock is persistent Update Time: Sat May 25 23:19:15 2013 State: clean, degraded, recovering Active Devices: 2 Working Devices: 3 Failed Devices: 0 Spare Devices: 1 Layout: left-blank Ric Chunk Size: 512 K Rebuild Status: 90% complete Name: xiaoluo: 0 (local to host xiaoluo) UUID: 029e2fe7: 8c9ded40: f5079536: d249ccf7 Events: 40 Number Major Minor RaidDevice State 0 8 16 0 active sync/dev/sdb 1 8 32 1 active sync/dev/sdc 3 8 64 2 spare rebuilding/dev/sde // new the/dev/sde hard disk has been added.