Linux Storage Management

At present the common disk type has Ide,scsi,sata,sas and so on, the IDE and SCSI is commonly known as the port, SATA and SAS are commonly known as the serial port, the two kinds of hard drives are common to personal computers and servers, the IDE's hard disk for a long time, performance is relatively low, SCSI is " Small computer system dedicated interface "short, SCSI hard disk is the use of this interface hard disk. SATA Hard drive A line only a piece of hard disk device, easy to achieve, standard low; SAS is the serial Port SCSI interface. General server hard disk use these two kinds of interface, its performance than the above hard disk to be higher, more stable, but the price is high, the capacity is small, the noise is big.

These kinds of hard disks run at different speeds, where the speed of the drive is the number of read/write IO per second;

IDE: Mechanical HDD, 50IOPS

SCSI: Mechanical HDD, 100-200iops

Solid-state drives, around 400

SATA: Mechanical hdd, around 100

Solid-state drives, around 400

SAS: Mechanical HDD, around 200

Solid-state drives, around 800

Here is a description of some of the necessary concepts of Linux storage management;

Mechanical HDD:

Track: Consists of one concentric circle after another;

Head (Heads): Each of the two sides of the magnetic plate has a head, a magnetic head corresponding to a piece of a face. Therefore, the first magnetic head can be used to represent the surface of the data.

Cylinder (Cylinder): Concentric tracks of the same radius in all the magnets constitute "cylinders", meaning that the series of tracks are stacked vertically together, forming a cylindrical shape. Simply understood, the number of cylinders = number of tracks.

Sector (Sector): divides the track into a number of small segments, which is the sector. Although very small, but actually a fan shape, it is called sector. The capacity of each sector is 512 bytes.

The standard of mechanical hard disk is the average seek time and disk speed, the average seek time is the time it takes to jump from one track to the other, accurate to NS, there are many factors affecting disk speed, such as calorific value, noise, etc.

Files stored in the dev directory under "/" are device files, files that store data, and are accessed as access points for certain devices and drivers;

The device has a character device with a block device:

Block device: Any device that can achieve data storage is a block device (U disk, etc.)

Character devices: Devices that enable data transfer and data processing, sequential access (mouse keyboard, etc.)

Device file Mode:

The storage file mode of the device file in the IDE hard disk is/dev/hd[a-d]

The storage mode of the device file in the Scsi/sata/sas/usb hard drive is:/dev/sd[a-z]

Reference to the device:

Devices are referenced by device names (such as/DEV/SDA) when storage management is performed on the device, and device filenames can often be referenced by the volume name and UUID;

How do I create a partition if I have an idle hard disk? How do I create a file system after I create a partition? How can I mount it?

To create a partition:

Partitions have two types of partitions, one is the MBR with hard disk space less than 2TB, and one is a GPT with hard disk space greater than 2TB

Mbr:

64bytes:partition table partitioned tables, each 16Byte indicates the contents of a partition, so it can represent four primary partitions altogether;

When the hard disk is divided into four partitions, regardless of the remaining disk space will not be able to create the partition, in this case, often choose to create three primary partitions and an extended space, the expansion space can create logical space in the space, continue to use the remaining space, so as to ensure the utilization of disk space; primary partition + The extended partition can only have four, the partition table is 1,2,3,4, the logical partition number starts from 5, regardless of the number of the previous four digits is occupied;

Gpt:

GUID Partition table,guid partition table, supports 128 partitions

UEFI (Unified extended Firmware Interface) hardware can support GPT

Partitioning tools:

FDISK: Used to create a management MBR partition, for a single drive, up to 15 partitions can be managed;

Gdisk: Used to create administrative GPT partitions;

Note: If you create a partition on a partitioned and mounted hard disk, even if it is saved and not read by the kernel, you can view the contents of the/proc/partitions text document to see if the kernel has read the altered hard drive, in which case the kernel can be forced to read the modified hard disk by several commands , or restart the computer, but restarting the computer is not easy in the first-line operation environment;

Read command:

partx-a [device]: If there is no device name, read all the hard drives in the current environment;

Partprobe [device]: If there is no device name, read all the hard drives in the current environment;

Fdisk

Fdisk-l: View the partition table of the current disk, and if you add the device name, view the partition table of the specified disk;

FDISK device Name: Enter the interactive mode of creating partition, in interactive mode can create partition, delete partition, view partition and other functions

Sub-command:

N: Add partition;

P: Create primary partition;

E: Create an extended partition;

L: Create logical partition;

P: Displays the current disk partition table;

D: Delete a partition;

M: Display Help;

Q: Do not save, exit edit

T: Modify the System ID of the partition (Swap conversion possible)

W: Write, save the modified operation to the disk partition table and exit, but do not read into the kernel, need to be enforced;

File system:

After partitioning the disk, you can create a file system on the disk partition, which is the logical address of the partition, the method and data structure on the partition, the process of creating the file system is actually the process of the format, the format is divided into two kinds,

The first is low-level format, that is, on the physical level of the file system to format the creation of tracks, etc.;

The second is advanced formatting, which is done after partitioning, the main role is to create a file system, for the partition to build a logical address unit, cluster, block, whether it is a cluster or block, is to organize one or more sectors together to complete the data storage; only one file content can be stored in a block if the file When the capacity is too large, multiple blocks can be stored;

The first step in creating a file system is to create several blocks with different effects:

Metadata block, data block, Block bitmap, Inode Bitmap, super block, log block, etc., let it make up a block group each block group has independent metadata blocks and data blocks, when storing data, the block group of data block first choice, if the block group of data block is not enough, you can use the storage policy to request multiple block group coordination work;

Metadata block (inode): The file system typically places data in separate chunks, permissions, attributes, file sizes, file owners, and pointers to the file's data blocks into the inode;

Data block: The actual data is placed into the block of the data block. The actual data refers to the contents of the file, and in the case of the data block, it also holds the inode number of its subordinate subdirectory, and finds its metadata block by inode number;

Block bitmap: A binary representation of which data blocks are empty, those data blocks are not empty, 0 is empty, and 1 is not empty;

Inode Bitmap: A binary representation of which inode data blocks are empty, which inode data blocks are not empty, 0 is empty, and 1 is not empty;

Super BLOCK: Defines how many block groups are, and the boundaries of each block group. In order to prevent the Super block failure, resulting in the entire system crash, the super block needs to be backed up;

Log block: Record the operation of data to prevent inconsistent data;

The file system that comes with Linux is:

Ext ext2 ext3 ext4: Maximum partition 50TB, can do with partition, can also do boot partition;

XFS: Enterprise-Class 64-bit file system, maximum partition 500TB, can do root and boot partition;

ReiserFS: Good anti-delete function;

Btrfs: Balanced tree file system;

Swap: Swap partition file system;

In fact, for different Linux versions, there is a default file system at the time of installation

such as Centos5:ext3

Centos6:ext4

Centos7:xfs

To create a file system:

Mkfs

MKFS.EXT3 Device Name (create Ext3 file system on a device)

If you are creating a file system for the EXT series, you can use the MKE2FS command

Note: The file system cannot be created directly on the disk when the file system is created, so that all partitions on the disk are formatted, and all logical partitions are deleted if the extended partition is used as parameters; only primary and logical partitions can be used as parameters;

MKE2FS:

Common options

-B block-size: Adjusts the size of the block when the file system is created;

-I byte-per-inode: Indicates the ratio of inode to byte, that is, how many bytes can be built with an inode, the default value is 16384,

-j: Create a file system with log function, clearly indicate that it is ext3

-N: Directly indicates how many inode numbers to create for this file system;

-m: Percentage of reserved blocks such as-M 1 (1% of disk space reserved for super users)

-T: Specifies the file system type EXT2,EXT3,EXT4, default is Ext2

-O feather[,..]: Turn off or turn on file system features;

-L: Specifies the volume label for the file system;

Such as

Mke2fs-t/dev/sda1 (default to Device/dev/sda1 create ext2 file system)

DUMPE2FS: File System Properties View tool, detailed list of file system various properties, such as the block and block groups of blocks of metadata block data block structure;

TUNE2FS: File system Property Adjustment tool, adjust those parameters that can be modified on the Ext file system;

Common options

-j: The Ext2 file system is promoted to ext3, but cannot be reversed

-m: Percentage of reserved blocks such as-M 1 (1% of disk space reserved for super users)

-O: Modify default mount options for the file system

-R: Modify the number of blocks of disk space reserved for super users

-L: Modify the file system's volume label;

-O: Modify the file system's characteristics;

-L: Displays information about the super block;

If you want to modify the file system's volume label information, you can use the E2label command;

E2label [Device] "volume label": Set the volume label information for the device;

E2label [device]: Displays the volume label information of the device;

Such as:

[Email protected] wjq]# e2label/dev/sdb1 "Wujunqi"

[Email protected] wjq]# E2LABEL/DEV/SDB1

Wujunqi

File System Detection Modification tool:

Fsck

-T Fstype: Specifies the type of file system to detect

-A: Automatically fix all problems without interaction

-R: Interactive fix issues

Ext Series File system specialized detection and repair tools:

E2fsck

-Y: All questions in the interactive process are answered with Yes

-F: Force detect fix, even if file system is in clean state

Blkid: Displays the device on which the file system was created, and the UUID of the device, the type of file system created;

Blkid Device Name

Blkid-l Label: The reference device can be searched by volume label according to the volume label;

Blkid-u UUID: The reference device can be searched by uuid, according to the UUID;

Such as

[Email protected] wjq]# Blkid

/dev/sr0:uuid= "2015-12-09-22-36-30-00" label= "CentOS 7 x86_64" type= "iso9660" pttype= "DOS"

/dev/sda1:uuid= "1b28e0b5-b2c5-4da2-bf77-70414e1424e4" type= "XFS"

/dev/sda2:uuid= "Ec3d759a-c817-41b6-9a37-dded7e985a26" type= "XFS"

/dev/sda3:uuid= "ec273085-9c8e-434c-bb15-0219217e9f4c" type= "swap"

[Email protected] wjq]# blkid-l "Wujunqi"

/dev/sdb1

Findfs: Find the corresponding file system according to the volume label and UUID

Findfs Label=label

Findfs Uuid=uuid

To create a swap file system:

The swap file system is created on the premise that the partition you create must be of type swap, or it can be transformed from the primary partition to the Fdisk interface, input T, and enter the code (swap sequence).

Mkswap command: Create swap file system

Common options

-L: Set the interactive partition volume label;

-F: Enforcement;

Such as

[Email protected] wjq]# mkswap-f/DEV/SDB3

[Email protected] wjq]# Blkid

/dev/sdb3:uuid= "4bf03dde-b724-43ff-b8b2-d3cd9d153bc6" type= "swap"

Using the file system-mount

   entire linux The system uses a directory tree schema, but our file data is actually placed in the disk partition slot, and now the question is "how to combine the architecture of the directory tree with the data in Disk"? -- mount; mount is to use a directory as the entry point, the disk partition slot data placed in the directory, that is, access to the directory can read the meaning of the partition slot. This action we call the mount "

Mount point: The location of the disk partition data, other files, can access the partition data through this mount point, the directory as a mount point must exist beforehand, and is not used by other processes, as a mount point of the original data, once mounted, its original data will be hidden;

DF Command: View mount information;

Options

-H: List disk space in the form of G,MB;

-T: Lists the types of disk space mounted file systems;

-P: Lists the number of blocks in the file system;

Free command: show memory space;

Options:

-K: Displaying memory space in kilobytes

-M: Displaying memory space in megabytes

-G: Uses 1 billion bytes to show memory space

FREE-M (displays disk space in megabytes)

Mount command

Mount

Common options:

Common options:

-r:readonly mount the target file system in a read-only manner;

-w:read and write, mount the target file system in a read-write manner;

-N: By default, the device mount or unload operation is updated synchronously to/etc/mtab, and if the-n option is used when mounting, it means that we want to disable this feature, even if the mount succeeds and is not updated to/etc/mtab;

/etc/mtab: A brief overview of the Mount command is recorded;

-T Vsftype: Indicates the type of file system created on the device to be mounted, and if omitted, the Mount command uses Blkid to determine the type of target file system to mount;

-L LABEL: Use the volume label instead of the device name when mounting the file system;

-U uuid: use UUID instead of device name when mounting file system;

-a:mount-a automatically according to the device that is filled in the/etc/fstab file

Mount;

-O Option:

Sync/nosync: Synchronous/Asynchronous operation, synchronization, as long as any data in memory is modified, it will immediately write such data to disk, increase data reliability, but will increase the consumption of Io, asynchronous, regardless of the data changes, at a later time to write the data to disk, reduce the consumption of IO;

Atime/noatime: Whether the access timestamp is updated when the file or directory is read access, but if the set atime consumes too much of the IO, a second is changed many times if there are many people accessing it;

Diratime/nodiratime: When the directory is read access, whether to update the access timestamp;

RO/RW: Mount file system in read/write-only mode

Dev/nodev: Whether to allow the creation of device files in this file system;

Exec/noexec: Whether the program files are allowed to run in this file system;

Auto/noauto: Whether to allow the use of Mount-a mode to Mount/etc/fstab

's file system;

User/nouser: Whether to allow ordinary users to mount the file system;

Suid/nosuid: Allow suid and Sgid special permissions on program files to take effect;

Relatime/norelatime: Whether to refer to the Inode access time to modify its modification time and change time;

Remount: Re-mount without unloading, you can apply the new hanging in the option;

ACL: can make file system support Facl function;

Default: Defaults, including: Rw,suid,dev,exec,auto,

Nouser,async,and,relatime

Default Nosuid ro (defaults is followed by the priority, then the defaults in defaults are executed;)

Loop: Using a loopback device

-B: Binds a directory to another directory, which is the same as the entry for another directory, the content of access to two directories is the same;

Mount-b Source Directory Destination Directory

Cases

Mount-o remount.ro mount point or device name

(Set read-only)

Loop: Using a loopback device

Xshell importing an ISO system file

Mount-o Loop import file path mount point

Such as:

To set readable writable:

[Email protected] wjq]# mkdir ooo

[Email protected] wjq]# MOUNT-RW/DEV/SDB5./ooo

[Email protected] wjq]# DF-TH/DEV/SDB5

File system type capacity used% mount point available

/DEV/SDB5 ext2 861M 888K 817M 1%/home/wjq/ooo

Mount according to the volume label name:

[Email protected] wjq]# mount-l Wujunqi./ooo

[Email protected] wjq]# DF-TH/DEV/SDB1

File system type capacity used% mount point available

/DEV/SDB1 ext4 19M 332K 17M 2%/home/wjq/ooo

Use of the-o option:

[Email protected] wjq]# mount-o ro/dev/sdb1./ooo

[Email protected] wjq]# DF-TH/DEV/SDB1

File system type capacity used% mount point available

/DEV/SDB1 ext4 19M 332K 17M 2%/home/wjq/ooo

/DEV/SDB1 on/home/wjq/ooo type EXT4 (ro,relatime,seclabel,data=ordered)

Unload mount point Command umount:

Umount Device Name

When unloaded, may be a user is in use, you can use the command lsof to see if there is a user, you can use the method of forced unloading;

FUSER-KM mount point (forced unload)

To mount a disc:

CD-ROM files are more special in/dev/sr0;

1. Ensure that the disc has the correct image file

2. Ensure that the optical drive is connected, select settings in the virtual machine option, and click the Connected option on the CD;

3. Create a CD mount point, mount/dev/sr0 mount point

Mount the USB flash drive: The system is identified beforehand and then mounted as normal;

To mount an interactive partition:

Swapon command:

-A: Automatically mount the interactive partition device stored in the/etc/fstab file;

The-a option in the Mount command above has the/etc/fstab text document mentioned, the six fields stored in the file can help us to set up the mount automatically, as long as the contents of the file can be automatically mounted;

The meanings of the six fields are:

UUID mount point path file system defaults dump frequency self-test order

Defaults: Default mount Options

Dump frequency: Use the dump command to backup the partition frequency 0: Indicates no backup 1, backup once a day 2, backup every other day, often 0;

Self-Test sequence: 0 is not self-test, 1 is a priority self-test, usually 0

Linux Storage Management