Linux storage management and hard disk partitioning, formatting, mounting

Here is the basics of storage management for Linux, using a new hard drive device step (partition, format, mount Partition).

First, Linux storage Management basic knowledge point

1. Hardware composition

In the computer basic knowledge, each hardware is represented as: the arithmetic Device (CUP), the controller (Meem), the hard disk (Hdisk), the network card (NIC, ie net interface card);

2, north-south bridge chip

The chip has the South Bridge and North Bridge two kinds of chips, South Bridge chip link Low speed, North Bridge Link Expressway, and the data will be received North bridge divergence. South Bridge Chip Link Optical drive, hard disk, etc. North Bridge chip link memory. Controller is generally located on the motherboard, is the interface after the implementation of the chip behind the function.

3. HDD

The hard disk mainly includes three parts: spindle, permanent magnet and head. The head can have one column, and its disk platters can have many slices, where data can be stored on both sides of the disk. Disk storage can be said to be magnetic disk generation.

4, five types of disk interface type

(1), IDE: For the same port, from ATA, transmission speed 133Mbps

IDE: Motherboard is on two interface, can connect up to 4 IDE hard disk

(2), SCSI: For the same port, dedicated server. The two actual types are: ultrascsi320:320mbps/ultrascsi640:640mbps

Scsi:7 interface, 15 HDD

(3), SATA (Serral ATA): Serial port. The actual three kinds are: SATA1, SATA2, sata3:6gbps

Sata:4-6 interface, 4-6 HDD

(4), SAS: For the serial port, the server uses, the transmission speed reaches 6Gbps

sas:16384 Block Hard Drive

(5), usb:1.0, 1.1, 2.0, 3.0 (480Mbps), 3.1 (10Gbps)

The disk interface uses dual-channel technology to improve transmission speed.

5. IOPS: Number of I/O per second

The following are the number of IOPS for five types of interfaces:

IDE: Mechanical hard disk, 50 IOPS

SCSI: Mechanical hdd, 100-200 IOPS

Solid-state drives, 400 or so IOPS

SATA: Mechanical hdd, about 100 IOPS

Solid-state drives, 400 or so IOPS

SAS: Mechanical hdd, about 200 ioPS

Solid-state drives, 800 or so IOPS

Huawei's PCI-E solid-state Drive IOPS: 100000+ (100,000) IOPS

6, mechanical hard disk related content

The basic terms of a mechanical hard drive are as follows:

Average seek time: The time the head is swapped from one track to another.

Disk speed (RPM): the speed at which the disk rotates around the spindle. Now the engine can reach up to 15000/s, the disk (hard disk) is vacuum encapsulated.

Track: Tracks

Sector: sector, sector is indicated by the arc length of the track edge, each sector size is 512Bytes, now the sector is actually mean

Cylinder: Cylindrical, projecting from track to all discs

Partition: Partitioning, which is a cylinder-based storage space

Head: Head, each disk surface has a head

7. Device File:/dev

Device files are access portals to drivers and devices that are associated to hardware devices.

Drivers generally refer to device drivers (Devices Driver), which are special programs that enable computers and devices to communicate. Equivalent to the interface of the hardware, the operating system only through this interface, to control the work of hardware devices, if a device driver is not installed correctly, it will not work properly.

Starting with RHEL6, CentOS6, all of the interface types of hard disk devices are named Sd[a-z].

Device number: The device number is the main device number and the second device number of two.

Major: The main device number, which distinguishes the device type, is used to indicate the driver required by the device.

Minor: The secondary device number, which distinguishes different devices under the same type, is the access entry for a particular device.

A person identifies a computer device file, and the computer identifies the device number. The main device number is the same device. The device file name, which complies with ICANN standards.

8. Mknod command

Mknod command: Mknod-make block or character special files

Mknod is in the format: Mknod [OPTION] ... NAME TYPE [MAJOR MINOR]

Option-M,--mode=mode to specify permissions

Example: # Mknod Mydev C 100

9, equipment files of two devices

(1), block equipment: For random access equipment, data storage and exchange units are blocks;

(2), character devices: devices used for linear access, the Exchange unit of data is generally a character.

10, the device file is stored under/dev/, the IDE interface named HD[A-D];SCSI/SATA/SAS/USB interface named Sd[a-z].

Starting with RHEL6, CENTOS6, all interface types of hard disk devices are named Sd[a-z]

Two device reference methods: Device file name, Volume label (Volume lable)

Use a new hard drive device step to: partition, create file system (format), Mount Partition.

Second, the division

1. Benefits of Partitioning

(1), optimize I/O performance:

(2), to achieve the quota limit of disk space;

(3), for high-speed repair;

(4), isolating system files and other program documents;

(5), install multiple operating systems.

2. How to Partition

(1), MBR partition

MBR partitioning: Used when hard disk space is less than 2TB, MBR partitioning is most commonly used. The MBR is the master boot record, which originated in 1982.

The MBR's 0 track 0 sector includes 512Bytes, which can be divided into three parts, as follows:

446 bytes:boot Loader, boot loader, default to GRU Bytes

bytes:partation table, partition table, each 168Bytes indicates the contents of a partition, therefore, a total of 4 primary partitions.

2 Bytes: End tag, usually 55AA

In the four partitions of the partitioned table, the first three is the primary partition, and the fourth bit extends the partition. There are many logical partitions in the extended partition, with only one logical partition. The extended partition cannot be used directly and must be partitioned two times.

Note: A, primary partition + extended partition, up to 4, the number of a partition table is followed by: 1, 2, 3, 4;

b, the logical partition number starts at 5, regardless of whether the preceding four numeric numbers are occupied.

(2), GPT partition

GPT partition: Used when hard disk space is greater than 2TB. GPT is the GUID partation table, the GUID partition tables, which support 128 partitions.

The UEFI (Unified Extended Firmware Interface) hardware can support GPT and boot faster.

3. Commonly used partitioning tools

(1), two kinds of partitioning tools

FDISK: For creating and managing MBR partitions, for a single hard disk, you can manage up to 15 partitions.

Gdisk: GNU disk. Used to create and manage GPT partitions.

(2), re-mount

If you use the Fdisk and Gdisk commands to partition the remaining space of a disk that has already been partitioned and mounted, the partition's information will not be re-read by the kernel, even if it is saved. If you want the kernel to recognize such partitions, there are several ways: (1), restart the computer, (2), Partprobe command, or PARTX command, forcing the kernel to reread the partition table, using the following:

centos5/7:partprobe-a [Device] indicates that the partition table is reread for all disks if the name is omitted.

centos6/7:partx-a [Device]

KPARTX-AF [Device]

The disk command that forces the kernel to recognize the SCSI interface that is plugged in on the boot state is as follows:

# echo '---' >/sys/class/scsi_host/host2/scan

(3), Fdisk partition example

FDISK command:

(Fdisk-manipulate disk partition table)

Fdisk-l [device ...] to view the partition table for the specified disk, or to display the partition table for all disk devices if the device parameter is omitted

FDISK device: partition management for the specified device.

The FDISK subcommand is as follows:

D Delete a partition

L list known partition types

M Print this menu

n Add a new partition

E Extended

P primary partition (1-4)

L Logical

P Print the partition table interchange partition tables in interactive mode

Q Quit without saving changes

t change a partition ' s system ID

W Write table to disk and exit

Example: Sequential input m-e-p partition

(4), parted command

(parted-a partition Manipulation program)

Parted:parted is GNU, there is an advanced partitioning tool that takes effect in real time, preferably without.

parted [options] [Device [command [Options ...] ...]]

Examples are as follows:

Parted/dev/sdb Mklabel Gpt|msdos

PARTED/DEV/SDB Print

Parted/dev/sdb Mkpart Primary 1 200

PARTED/DEV/SDB RM 1

Iii. Creating a file system (formatting)

The file system consists of the following three aspects: FHS, permissions, naming rules for files

1. Why should I create a file system?

Creating a file system is essentially a logical addressing of storage space for the partition. Not creating a file system can also be stored, just read more complex.

Take the book's Plane classification management as an example, want to search for the need of books, no special sorting words need to traverse. If the spatial classification of books management, that is, divided into categories, which can be nested within each other, this is a few limited but effective management.

2, the format is divided into two categories: low-level format and advanced format.

Low-level formatting: mainly in the physical level of the completion of the format, when the disk device factory, through the low-level format to create tracks. The main function is to divide the tracks.

Advanced formatting: This is done after partitioning, the main role is to create a file system, for the partition to build a logical address unit, also known as a cluster or block, but whether it is a cluster or block, is to organize one or more sectors together to complete the data storage.

Windows cluster Size: 512Byte, 1024Byte, 2048Byte, 4096Byte, 8192Byte, 16384Byte, 32768Byte, 65536Byte

Size of Linux blocks: 1024Byte, 2048Byte, 4096Byte

Note: (1), a block can only place the contents of one file, but a block may occupy more than one block.

(2), if in a partition, will create a large number of small files, the block partition smaller; Conversely, the block is larger.

(3), if the partition space is relatively large, the block should be larger.

3, two function blocks

A file system is created in a partition: The partition is first divided into blocks of equal size, divided into two function blocks, metadata blocks and data blocks. The metadata in the metadata block is index (index node), which holds the fixed storage space for the metadata, called the inode. Each inode has a corresponding inode number. Data blocks are used to store streaming data in a file.

File attributes for metadata include file size, file permissions, file owner, file timestamp, file block pointer.

Pointers are divided into three categories: direct pointer, indirect pointer, level two indirect pointer

4. Special Documents:

(1), device and symbol link file

Device files: Block special or character special

The device file is simply replacing the data pointer in the metadata with the device number.

Example of a symbolic link file: Symbolic link

A symbolic link file simply replaces the data block pointer with the access path of another file.

In special files, this type of file occupies only the metadata region and no data blocks are occupied.

(2), bitmap bitmap

Bitmap bitmap: Use binary to store data block usage, 1 for already used, 0 for idle. Bitmaps are divided into block bitmaps and meta-data bitmaps, block bitmap and Inode bitmap.

(3), grouping and dividing blocks

Grouping in chunks, generating block groups, metadata in block groups, and metadata for block groups.

With more and more disk space, we divide the number of fast more and more, management is very inconvenient, so, block groups can solve the problem, each block group has a separate metadata block and data block, when we store data, block group of data block first choice, if the fast rent within the data block is not enough, Storage policies can be used to co-store multiple block groups.

Super BLOCK: The main definition is how many block groups, and the boundaries of each block group.

Log blocks: Blocks that record data for write operations, preventing data inconsistencies. Inconsistent data is usually caused by an unclean shutdown or an unclean power outage.

A complete file system can include: Super block, Inode, Inode Bitmap, block bitmap, data block, journal block.

(4), example: What is the process of trying to find the contents of a/var/log/messages file?

First, find/catalog Inode number, find the corresponding inode from the data region, then read the data block pointer in the Inode, find the data block of the contents of the storage/directory;

Then, in the data block to find the name var file name, if any, locate its inode number, in the metadata region to find the corresponding number of inode, and then according to the data block pointer to find the contents of the Var directory to save the data block;

Then, in the data block to find the name of log file name, if any, locate its inode number, in the metadata region to find the corresponding number of inode, and then according to the data block pointer to find the data block to save the log directory contents;

Finally, find the file name named messages in the block, and if so, locate its inode number, locate the inode in the metadata region, and then, based on the block pointer, find the data block that holds the contents of the messages directory, and you can read the data.

Linux-supported file systems

The Linux file system consists of two parts, the file system driver located in the kernel space and the file System management application located in the user space.

File system driver in kernel space: It is part of the kernel and can be made into kernel modules.

File System management applications in user space: A series of administrative commands.

5, Linux's own file system

Linux's own class four file system is as follows:

(1), ext, ext2, ext3, Ext4

The maximum partition that the EXT4 can reach is 50TB, and the partition can do the root partition, or it can do the boot partition.

(2), XFS

XFS is currently an enterprise-class 64-bit file system with a maximum partition of 500TB, which can be used as the root partition and boot partition.

(3), ReiserFS

ReiserFS has a good anti-delete function

(4), Btrfs (Balanced tree file system)

Btrfs is a technical preview, not particularly mature.

For different distributions Linux, there are default file system choices, as follows:

RHEL5 and Centos5 correspond to ext3;

RHEL6 and CENTOS6 correspond to EXT4;

RHEL7 and Centos7 correspond to XFS.

Disc File system: iso9660

Network File system: NFS, CIFS

Cluster file system: GFS2, OCFS2

Kernel-level Distributed File system: Ceph

Pseudo file system: Proc, SYSFS, Tmpfs, hugepagefs

File system for Windows: VFAT, NTFS

UNIX File systems: UFS, Fffs, JFS, ZFS

Interactive partition file system: Swap

Distributed file systems for user space: MogileFS, Moosefs, Glusterfs

Attention:

Do not use the disk device name and extended partition device name as parameters for the MKFS command, if you use a disk device as the parameter of MKFS, all partitions will be deleted, and if the extended partition is used as the MKFS parameter, all logical partitions will be deleted. The device that is capable of mkfs command parameters is the primary partition device and the logical partition device. An extended partition cannot be formatted and cannot be formatted such as a SDB disk.

6. Various tools for managing file systems

Tools for creating File systems: Mkfs (MKFS.EXT2, Mkfs.ext3, MKFS.EXT4, MKFS.XFS, Mkfs.btrfs 、......)

File system detection and Repair tools: fsck (FSCK.EXT2, Fsck.ext3, FSCK.EXT4, Fsck.xfs 、......)

File System Properties View tool: DUMPE2FS

File system Property Tuning tool: Tune2fs

7. Create file System Tools: Mkfs, MKE2FS, E2label, TUNE2FS

(1), Mkfs command

(Mkfs-build a Linux filesystem)

MKFS format: mkfs [Options] [-t type] [fs-options] device [size]

Example: Mkfs-t ext2/dev/sdb1

Note: A, do not use the disk device name and the extended partition device name as parameters for the MKFS command. If you use a disk device as a MKFS parameter, all partitions will be deleted, and if you use an extended partition as the MKFS parameter, all logical partitions will be deleted

b, can be used as MKFS command parameters of the device, is the main partition equipment and logical partition equipment;

(2), Mke2fs command

(Mke2fs-create an EXT2/EXT3/EXT4 filesystem)

You can use the MKE2FS command if you are only planning to create the Ext series file system.

MKE2FS common options are as follows:

-B block-size: When creating the file system, adjust the block size;

-I bytes-per-inode: Indicates the ratio of inode to Byte, that is: How many bytes can be built an inode, the default value is 16384;

-j: Create a file system with log function, that is ext3

-N number-of-inodes: Directly indicates the number of inode to be created for this file system;

-M Reserved-blocks-percentage: Specifies the percentage of disk space reserved for superuser root; default is 5;

-T Fstype: Specifies the file system type, ext2 ext3 ext4, default is Ext2

-O [^]feature[,...] : Turn off or turn on file system features

-L Volume-label: Assigning a volume label to the file system

(3), E2label command

(E2label-change the label on an EXT2/EXT3/EXT4 filesystem)

E2label format: E2label device [New-label]

You can use the E2label command if you plan to "modify the volume label for the Ext series files" only.

Note: If the volume label information is not added, the volume label of the specified device is viewed, and if the volume label information is added, the volume label of the specified device is modified.

(4), TUNE2FS command:

Tune2fs-adjust tunable filesystem Parameters on Ext2/ext3/ext4 filesystems (adjust those modifiable parameters on the ext2,3,4 file system)

If you want to modify the properties of the Ext series file, you can use the TUNE2FS command

The common options for the TUNE2FS command are as follows:

-j: Modify the Ext2 file system to ext3;

-M Reserved-blocks-percentage: Modify the percentage of disk space reserved for super users;

-O [^]mount-options[,...] : Modify the default mount options for the file system;

The ACL under-o Enable Posix Access Control Lists.

-R Reserved-blocks-count: Modify the number of blocks of disk space reserved for super users;

-L Volume-name: Modifying the file system's volume label

-O [^]feature[,...] : Modify the characteristics of the file system to turn it off or enabled;

-L: Displays the contents of the Super block equivalent to Dumpe2fs-h

8. File System detection and Modification tool

(1), fsck tool

The FSCK tools and usage options are as follows:

-T Fstype: Specifies the type of file system to detect

-A: Automatically fix all problems without interaction

-R: Interactive fix issues

(2), e2fsck tools

"EXT series file systems have specialized detection and repair tools"

E2FSCK tools and usage options are as follows:

-Y: All questions in the interactive process are answered with "Yes"

-F: Force detect fix even if file system is in "clean state"

9. Blkid command

Blkid-locate/print Block Device attributes

Blkid-l Label | -U UUID

Blkid-command-line utility to Locate/print block device attributes

Blkid-l Label: Search by volume label

Blkid-u UUID: Search by UUID

10. Findfs command

(Findfs-find a filesystem by label or UUID)

Findfs Label=label

Findfs Uuid=uuid

Example: Findfs "Label=mylable"

10. Create swap file system

Prerequisite: The partition type must be a swap type.

The modification is done by using the T command to specify the type of partition to be modified to 82 in Fdisk interactive mode.

Mkswap command:

(Mkswap-set up a Linux swap area)

Format: [options] device [size]

-L LABEL: Sets the volume label for the interactive partition

-F: Forced execution

Iv. using the file system (Mount)

1, Mount point

Other file systems other than the root filesystem, if they want to be accessed, must somehow be associated to the root filesystem by loading other file systems into a directory on the root system, a process known as "mount." The directory that is used to correlate other file systems, called "mount points"

Mount points: Mount point, which is also an access entry for other file systems, for example:/boot

mount point requirements: (1), the directory that is the mount point must exist beforehand

(2), as a mount point directory should be not used or can not be used by other processes directory

(3), as a mount point directory once mounted file system, its internal original files will be hidden

2. Complete mount operation, use Mount command

Mount Command:

(Mount-mount a filesystem)

Mount [-FNRSVW] [-t vfstype] [-O options] Device dir

Common options:

-r:readonly, mount the target file system in a read-only manner;

-w:read and write, mount the target file system in a read-write manner;

-N: By default, the device mount or unload operation is updated synchronously to/etc/mtab, and if the-n option is used when mounting, it means that we want to disable this feature and will not be updated to/etc/mtab even if the mount is successful.

-T Vsftype: Indicates the type of file system created on the device to be mounted, and if omitted, the Mount command uses Blkid to determine the file system type of the target device to be mounted;

-L LABEL: Use volume label instead of device name when mounting file system

-U uuid: use UUID instead of device name when mounting file system

-a:mount-a: Automatically mount according to the equipment that is filled in the/etc/fstab file;

-O Option:

Sync/nosync: Synchronous/Asynchronous operation

Atime/noatime: Whether the access timestamp is updated when the file or directory is read access;

Diratime/nodiratime: When the directory is read access, whether to update the access timestamp;

RO/RW: The file system is mounted in read/write mode;

Dev/nodev: Whether to allow the creation of device files in this file system;

Exec/noexec: Whether the program files are allowed to run in this file system;

Auto/noauto: Whether the file system in the/etc/fstab is allowed to be mounted using the Mount-a method;

User/nouser: Whether to allow ordinary users to mount the file system;

Suid/nosuid: Allow suid and Sgid special permissions on program files to take effect;

Relatime/norelatime: Whether to refer to the Inode access time to modify its modification time and change time

Remount: The new Mount option can be applied without the uninstall implementation.

ACL: can make file system support Facl function;

Defaults: Default options include: RW, suid, dev, exec, auto, nouser, async, and Relatime

Loop: Use the loopback device, mount the image file to the mount point;

Example: Defaults,ro,nosuid

-B,--bind: Binds a directory to another directory;

Mount--bind|-b Source Directory destination Directory

3. unmount file system: Umount Mount_point|device

4. View mount Information: DF-H/-HT/-HTP

5. Mount the disc as follows:

(1), ensure that the CD image file into the virtual machine CD-ROM

(2), to ensure that the optical drive is in a "connected" state

(3), create a CD mount point, for example: Mount-r/dev/sr0 mount_point

6. Mount the USB stick:

You need to have the system identify the USB flash drive device, and then mount it according to the device name; for example:/DEV/SDC1

7. Mount the Interactive partition

(1), Swapon, Swapoff command

(Swapon, swapoff-enable/disable devices and files for paging and swapping)

-A: Automatically mount the interactive partition device stored in the/etc/fstab file;

(2), DF command

The option can be:-H-T-P, which can be combined to write.

(3), free command

The options can be:-k-m-G.

8. Format of/etc/fstab file

The format of the/etc/fstab file is a total of 6 fields, separated by white space characters, as follows:

(1), the device to be mounted

Device file name, LABEL, UUID, pseudo file system

(2), Mount point

Swap devices do not require a mount point, just specify swap.

(3), File system type

(4), Mount option: Defaults

(5), Dump frequency:

How often to back up partitions using the dump command

0: No Backup

1: Backup Once a day

2: Backup every other day

(6), self-test order

0: No self-test

1: Priority self-test, usually only set on the root file system;

2: Secondary self-test

...

9: Lowest priority self-test

Linux storage management and hard disk partitioning, formatting, mounting