Multi-path usage and high-capacity hard drive mounts for Linux in Oracle database

First, the application of the environment and demand

1. Operating system is red Hat 6.5 64-bit

2. Mount storage capacity is 2.5T

Based on this application environment, there are two issues that need to be addressed:

1. In order to ensure the stability and transmission performance of the link, multi-path technology can be used.

2. Mount storage hard drive exceeds 2T,MBR partition format cannot be supported, need to use to GPT partition format

Second, what is multi-path

Excerpt from: http://rootking.blog.51cto.com/2619611/476212

The normal computer host is a hard disk attached to a bus, here is a one-to-one relationship. In the case of a SAN environment with optical fiber, or an ipsan environment made up of iSCSI, the host and storage are connected by a fibre switch or multiple network cards and IP, which makes the relationship much more numerous. In other words, host-to-storage can have multiple paths to choose from. The IO between host-to-storage is selectable by multiple paths. Each host to the corresponding storage can go through a few different paths, if it is used at the same time, how I/O traffic allocation? One of the paths broke down, how to deal with it? Also in the operating system perspective, each path, the operating system will be considered to be an actual physical disk, but in fact only to the same physical disk of the different paths, which is in use, the user is confused. Multi-path software is to solve the above problems arose.

The main function of multipath is to work with storage devices to achieve the following functions:

1. Failover and recovery of faults

Load Balancing of 2.IO traffic

3. Virtualization of Disks

Because Multipath software is needed and stored together with the use of different vendors based on different operating systems, the different versions are available. And some vendors, software and hardware are not sold together, if you want to use multi-path software, you may also need to buy license to the manufacturer. For example, EMC's multi-path software under Linux requires a separate purchase of license. Fortunately The 2.6 kernel of Redhat and SuSE comes with free multipathing packages and is free to use, as well as a more generic package that supports most storage vendors ' devices, even if some are not famous, and can be supported and run nicely by making minor modifications to the configuration files.

The more intuitive feeling is that the command is executed in a Linux system, and a fdisk -l hard disk similar to/DEV/SDA1,/DEV/SDB1,/DEV/SDC1,/DEV/SDD1 will appear. Because there are four combinations of paths, the Linux system will assume that each hop link is mounted on a single drive.

Three, Linux under Multipath Introduction

In Red hat, Multipath is already installed by default:

[Email protected] ~]# Rpm-qa|grep mapperdevice-mapper-multipath-0.4.9-64.el6.x86_ 64device-mapper-event-libs-1.02.77-9.el6.x86_64device-mapper-multipath-libs-0.4.9-64.el6.x86_ 64device-mapper-persistent-data-0.1.4-1.el6.x86_64device-mapper-libs-1.02.77-9.el6.x86_ 64device-mapper-event-1.02.77-9.el6.x86_64device-mapper-1.02.77-9.el6.x86_64

Among them (excerpt from: http://rootking.blog.51cto.com/2619611/476212):

2. Device-mapper-multipath: That is multipath-tools. Mainly provide MULTIPATHD and multipath tools and multipath.conf and other configuration files. These tools create and configure multipath devices through the Ioctr interface of the device mapper (call Device-mapper's user space library and create a multipath device in/dev/mapper).

4. Device-mapper: Mainly consists of two parts: the kernel part and the user part.

   The kernel part consists mainly of device mapper core (Dm.ko) and some target driver (Md-multipath.ko). The core completes the mapping of the device, and target handles the I/O from the mappered device, based on the mapping relationship and its own characteristics. At the same time, in the core section, provides an interface, the user through the ioctr can communicate with the kernel, to guide the kernel-driven behavior, such as how to create mappered device, these divece properties and so on. The user space portion of the Linux device Mapper mainly includes the Device-mapper package. These include the Dmsetup tool and some libraries that help you create and configure mappered device. These libraries are primarily abstract and encapsulate interfaces to ioctr communication to facilitate the creation and configuration of mappered device. These libraries need to be called in the Multipath-tool program.

6. Dm-multipath.ko and Dm.ko:dm.ko are device mapper drivers. It is the foundation of realizing Multipath. Dm-multipath is actually a target driver for DM.

8. SCSI_ID: Included in the Udev package, you can configure the program in multipath.conf to get the serial number of the SCSI device. By ordinal, you can tell that multiple paths correspond to the same device. This is the key to multi-path implementation. SCSI_ID is to query the identity of the SCSI device by sending the inquery command EVPD page80 or page83 to the device via the SG driver. However, some devices do not support EVPD's inquery commands, so they cannot be used to generate multipath devices. However, scsi_id can be rewritten to provide a virtual identifier for a device that cannot supply a SCSI device identity and output to standard output. When the Multipath program creates the Multipath device, it calls SCSI_ID and obtains the SCSI ID of the device from its standard output. When overwriting, the return value of the SCSI_ID program needs to be modified to 0. Because in the Multipath program, the straight is checked to determine if the SCSI ID has been successfully obtained.

Iv. Configuration Multipath

The basic configuration script is as follows:

[[email protected] ~]# cat /etc/multipath.confdefaults {       polling_interval       30       Failback               immediate       no_path_retry           queue      rr_min_io               100      path_checker            tur      user_friendly_names     yes      }      # SVC             device {                  vendor                    "IBM"                   product                   "2145"                   path_grouping_policy    group_by_prio                  prio_callout              "/sbin/mpath_prio_alua /dev/%n"                                  }

Multipath Basic Operations Command

#/ETC/INIT.D/MULTIPATHD Start #开启mulitipath服务 # multipath-f #删除现有路径 # multipath-v2 #格式化路径 # multipath-ll #查看多路径

If the configuration is correct, the/dev/mapper/directory will be more than MPATHBP1 and other devices, with the fdisk -l command can see the Multipath software created by the disk, such as:/DEV/MAPPER/MPATHBP1

V. Format the hard drive

Execution fdisk -l , you can see that the storage has been identified successfully, and that the multipath configuration is correct. The information is as follows:

[[Email protected] ~]# fdisk -l ...] disk /dev/mapper/mpathb: 2684.4 gb, 2684354560000 bytes heads, 63  sectors/track, 326354 cylindersunits = cylinders of 16065 * 512 =  8225280 bytesSector size  (logical/physical): 512 bytes / 512  bytesi/o size  (Minimum/optimal):  512 bytes / 512 bytesdisk identifier:  0x00000000             device boot       Start         End       Blocks   Id  System/dev/mapper/mpathbp1                1       267350  2147483647+  ee  gpt ...

The above information can be found in the GPT partition format, the next step is to format the hard disk. If not, you need to perform the following steps first:

1. Create a new partition

[[email protected] ~]# pvcreate /dev/mapper/mpathb[[email protected] ~]#  Parted /dev/mapper/mpathbgnu parted 2.1using /dev/mapper/mpathbp1welcome to gnu  Parted! Type  ' Help '  to view a list of commands. (parted)  mklabel gpt                  #设置分区类型为gpt (parted)  mkpart extended 0% 100%     #扩展分区 and use the entire hard drive (parted)  quit                         #退出                                                   information: you&nBsp;may need to update /etc/fstab. 

2. Format & Mount HDD

[Email protected] ~]# mkfs.ext4/dev/mapper/mpathbp1[[email protected] ~]# mount/dev/mapper/mpathbp1/test

After the mount is successful, it can be used.

3. Dynamic Mount Partition

When a new partition is created in the system, because mount mounts fail after rebooting the system, the partition information needs to be written to the/etc/fstab file to be permanently mounted.

[[email protected] ~]# vi/etc/fstab/dev/mapper/mpathbp1/test EXT4 defaults 1 2

Save exit,/DEV/MAPPER/MPATHBP1 will automatically mount to/test directory after reboot

Multi-path usage and high-capacity hard drive mounts for Linux in Oracle database