Linux basics [Article 9] and linux basics Article 9

Linux basics [Article 9] and linux basics Article 9

Architecture of adding three disks to the server

 

Note:

1. linux caches the physical memory that is not used by the system. Therefore, 859 is not the actual memory of the system.

2. The real available system memory is 921.

3. buffers is the write buffer, and sync writes the buffer data to the disk.

4. the cache is the buffer for reading data.

5. The hard disk is mechanical, and reading is too slow, so reading and writing are used for caching technology.

6. Portal architecture websites use cache technology to prevent users from accessing the disk as much as possible.

Internal Structure of the Disk

The internal structure of the disk mainly includes disk, Head, disk spindle, control motor, head controller, data converter, interface, cache, and so on.

Head Assembly and head Drive Device

1. head assembly. It consists of three parts: read/write head, drive arm, and drive shaft.

The principle of reading data from a disk is to use the polarity of a specific magnetic particle to record data. When reading data, the head converts the polarity of the magnetic particles into different electrical pulse signals, and then uses the data converter to convert these original signals into data that can be used by the computer. The write operation is the opposite.

2. Head drive device. The disk seek relies on a mobile head, which must be driven by the device (mechanical operation ). The head Drive Device consists of an electromagnetic coil motor, a head drive car, and a anti-vibration device. The high-precision Light head drive device can correctly drive and locate the head, in addition, it can precisely locate the channels specified by system commands in a short period of time.

3. disks. Disks are the real carrier of data stored on disks. Most disk disks use metallic film (or plexiglass ), compared with floppy disks, this metal film has higher storage density, high residual magnetism, and high coercive force.

4. Spindle components. Spindle components include bearings and drive motors. The number of turns on the disk spindle is an important reference for measuring disk read/write performance. For example, SAS15K and SATA10K.

5. Pre-control circuit. The front circuit controls the signal induced by the head, the speed regulation of the spindle motor, the head drive and the servo positioning. Because the signal read by the head is weak, sealing the amplification circuit in the cavity can reduce external signal interference, improve the accuracy of operation commands.

Disk selection during Enterprise Production

The current server market is dominated by SAS, SATA, and SSD disks.

Enterprise SAS hard drive

The common SAS hard drive in enterprises is 15000 rpm. The current mainstream 300G, 600G, 1 T, from the specific business needs and cost-effectiveness considerations, work with a-g sas hard drive. It is used to provide general external service servers on the production line. For example, the Database Service, storage service, image service, and related high-concurrency Service (web http and cache service) on the production line. In general, if there are no special business requirements, SAS disks are the preferred disk configuration in the production environment.

Enterprise-level SATA hard drive

Enterprise-level SATA hard drive-rpm, with a common capacity of 1 TB, 2 TB, 4 TB, and 6 TB. The advantage is cost effectiveness and high capacity. Considering the specific business needs and cost-effectiveness, in work, SATA disks are used for offline data storage or concurrent business services that do not provide services, such as offline backup of site programs, databases, and images. High Cost Performance, generally 2 tb sata disks.

Disk purchase highlights:

1. SAS disks are used for online businesses.

2. Offline services use SATA disks and tape libraries.

3. SSD disks are used for online businesses with high concurrency and small capacity.

4. intelligently analyzes Hierarchical Storage Based on Data Access popularity.

Note: Do not use SATA disks for data storage or database services for Online High concurrency services.

Disk terminology

English	Chinese
Disk	Disk
Head	Head
Sector	Slice
Track	Track
Cylinder	Cylindrical
Units	Unit block (the size of a cylinder)
Block	Data Block
Inode	Index Node

When the disk does not work, the head stops at the surface near the contact disc of the spindle, that is, the place with the smallest wire speed. This is a special area that does not store any data, it is called the start and stop zone or the landing zone. The data zone is not the start and stop zone.

In the outermost ring of the disk, the farthest track from the spindle is called the "0" track. The disk data is stored starting from the outermost ring "0" track. The "0" track is very important. The system boot program is in the first 446 bytes of the 0-cylinder 0 track 1 sector.

Track

Each disk has two sides and information can be recorded. The disc surface is centered around the disc. It is called a magnetic track to record the circular magnetization of different radius of data.

The disk is divided into many concentric circles during formatting. the trajectory of These concentric circles is called track ). The track starts from 0 in sequence from the disk surface to the inside.

Cylindrical

The circular trajectory of the track with the same radius on all disks is sequentially formed into a cylinder ), the head of each cylinder starts from "0" from top to bottom.

Number of cylinders = number of tracks on a disk

Slice

The disc has a straight line from the center of the circle to the surrounding area. Different magnetic channels are divided into multiple slice (ARC) areas. Each Arc area is called a slice, and each slice is 512 bytes in size.

The operating system stores information on disks in units of sectors (sector). Generally, the size of each sector is 512 bytes. A slice consists of two parts: the identifier of the Data Location and the data segment that stores the data.

The first major part of a sector is the identifier. An identifier is the header of a sector. It consists of three numbers that constitute the three-dimensional address of a sector, the head (or disk) of the sector, the track (or the cylinder number) and the position of the slice on the track, that is, the fan area number. The bidding also contains a field that shows whether the slice can store data reliably or whether a fault has been found and thus is not suitable for use. Some disk controllers also record indicators in the Sector Header to guide the disk to the replacement sector or track when an error occurs in the original sector. Finally, the Sector Header Mark ends with the cyclic redundancy check (CRC) value for the Controller to check the reading status of the Sector Header mark and ensure that the preparation is correct.

The second major part of the slice is the data segment that stores data, which can be divided into data and the Error Correction Code (ECC) that protects the data ). During the initial preparation, the computer fills in this section with 512 virtual information bytes (actual data storage location) and ECC numbers corresponding to these virtual information bytes.

Summary of the cylindrical sectors of the track

The most basic components of a disk are disks coated with magnetic media (with many layers) made of hard metal materials. The number of disks varies with disk capacity.

1. A disk has 2-14 disks, each of which has two sides. Each side should have a read/write head. The head number is used to distinguish the disk surface. That is, the number of disks is the number of cores, disk quantity x 2 = number of cores (Disk quantity ).

2. The channels on different disks are divided into multiple slice areas, each of which is a sector ).

3. On the same disc, the center of the disc is the center. A circular track with different radius is a track ).

4. A cylinder is a cylinder formed by a track with the same radius on different disks ).

5. One cylinder contains multiple channels with the same radius and one track contains multiple sectors.

6. Data Information records can be expressed as a head, a track (cylindrical), and a sector.

Disk Capacity Calculation

Method 1:

Disk capacity = 512 * number of sectors * Number of tracks * Number of cores

Size of each track = 512 * number of sectors

Size of a disk = 512 * number of sectors * Number of tracks

Method 2:

Disk capacity = Number of cylinders * size of the cylinder

Number of cylinders = number of tracks

Cylindrical size = track size * Number of cores

Track size = 512 * number of sectors

Number of cores: 255 heads, number of sectors: 63 sectors/track, Number of cylinders: 25.

Disk size = 512 * number of sectors * Number of tracks * Number of cores = 512*255*63*25 = 205632000 bytes

Mechanical disk read/write principles

1. the disk reads and writes data based on the cylinder. That is, it first reads a track on the same disk. After reading the data, if the data is not read, the head will not switch to other tracks, instead, you can choose to switch the head to read the tracks with the same radius on the next disk. If the data has not been read or written, to switch to another track with different radius. The process of switching the track is called seeking.

2. switching between different heads is electronic switching, while switching between tracks requires radial movement of the head. This radial movement requires stepping motor adjustment, which is a mechanical switching.

Disk Partition logical structure of disk storage

The storage structure of a hard disk includes the Master Boot Record, partition table, partition end mark, and other Partition Boot Information and Data zone information.

The primary Boot Sector is unique in the entire hard disk, that is, one hard disk can only have one. Below is a magnified image of the zero track and one sector.

16-byte Partition Table content

Bytes	Description
1 bytes	State: partition status, 0 = inactive, 0x80 = activated.
1 bytes	StartHead: Start head number of the partition
2 bytes	StartSC: Start sector and cylindrical Number of the partition. The lower 6 bits in the lower byte are the sector number, the higher 2 bits are the 9th and 10th bits of the cylindrical number, and the higher byte is the lower 8 bits of the cylindrical number.
1 bytes	Type: partition type, such as 0x0B = FAT32, 0x83 = linux. 00 indicates that this item is not used.
1 bytes	EndHead: the end head of the partition.
2 bytes	EndSC: End slice and Cylinder Number of the partition
4 bytes	Relative: Address of the Relative sector of the partition in linear addressing mode (absolute address for the basic partition)
4 bytes	Sectors: partition size (total number of sectors)

Backup and recovery MBR

Disk Partition

1. The essence of disk partitioning is to set the Partition Table of the next 64 bytes after the first 446 bytes of the first bytes of the 0-core 0-Track 1 sector, that is, to divide the start and end head numbers, and Fan area and cylinder.

2. The partitioning tool has fdisk (suitable for Disk Partitions smaller than 2 TB), parted (suitable for Disk Partitions larger than 2 TB, and smaller than 2 TB), and fdisk is preferred, parted is selected only when the value is greater than 2 TB.

3. The partition table of a disk is only 64 bytes in size, and each partition table occupies 16 bytes. Therefore, a disk only supports four partition table information, that is, the total number of primary partitions and extended partitions cannot exceed 4.

4. disk partitions are divided by cylinder.

5. You cannot directly use extended partitions. You also need to create logical partitions Based on the extended partitions.

6. Extended partitions have their own partition tables. Therefore, there can be multiple logical partitions under the extended partitions.

A disk usually needs to be partitioned before it is used. Of course, it can be used directly without partitioning, but this is not a common case. Disk Partitions include primary partitions, extended partitions, and logical partitions. A hard disk can have up to four partition table information (limited by the disk itself). The location of a primary partition can be replaced by an extended partition, in addition, a hard disk can only have one extended partition (Operating System Restrictions). In this extended partition, multiple logical partitions can be divided (the IDE disk partition number is about 5-63 ), SATA (serial number 5-15 ).

A server has 6 GB disks. After RAID 5, the total size is 3 TB. The solution is to install the system on the raid 5 instead of restarting the system directly. Instead, on the raid interface, the system will be mounted on a small Virtual Disk (vd) of 200 GB, after the system is installed, the remaining 2.8T is partitioned by parted.

Primary partition primary

A primary partition is a required partition on a disk. It is generally the first partition on the disk. A disk can have up to four primary partitions.

A disk is divided into six partitions, the number of primary partitions and the number of extended partitions:

1 p + 1e, 2 p + 1e, 3 p + 1e.

Extended partition Extended

The extended partition is like a virtual small hard disk, but the difference is that there is no MBR, but only the extended partition table, and this extended partition indicates there is no limit of 64 bytes, therefore, multiple logical partitions can be divided on the extended partitions. A disk can only store one extended partition, and the extended partition cannot store data directly. The extended partition is limited by the operating system.

Logical partition logical

You cannot directly partition a logical partition (fdisk) in a disk. A logical partition must exist in an extended partition. You can divide multiple logical partitions in an extended partition. The number of the logical partition starts from number 5. Multiple logical partitions can be divided in this extended partition (the approximate number of the IDE disk can be 5-63 ).

Disk Partition considerations

1. Any number of primary partitions, but 1 <= Number of Primary partitions <= 4.

For example, a hard disk can be divided into four primary partitions, three primary partitions, two primary partitions, and one primary partition.

2. Extended partitions can be combined with primary partitions, but 2 <= (primary partition + extended partition) <= 4 is required.

For example, 3 primary partitions + 1 extended partition, 2 primary partitions + 1 extended partition, or 1 primary partition + 1 extended partition. When the total number of partitions is greater than four, an extended partition must be divided in advance.

3. If you want to partition data into four disk partitions, you can use the partitioning method p + p or p + e.

Note:

Due to the MBR and disk partition table size restrictions, if you allocate 4 p or 3 p + e to the disk, even if the disk has no available space, therefore, this part of the space cannot be used as a partition (because no partition can be divided, the partition table space is already full ). Therefore, when planning a partition, if the business requires more than four partitions, you can select the 3 p + e partition method, in addition, in the last extended partition, all the remaining space should be allocated to this extended partition. Of course, you can also use 1 p + 1e and 2 p + 1e partitions. In fact, the primary partitions cannot be fully divided, but the primary partitions are not retained.

Note:

1. An extended partition is not a real available partition. After an extended partition is created, a logical partition must be created on the extended partition before it can be used.

2. There is no difference between primary partitions and logical partitions in the use of one data storage (for most data storage). When installing the operating system, select the primary partition as the first partition.

3. The Partition Number 1-4 is reserved for the primary partition or extended partition. The logical Partition Number can only start from 5 and will not be allocated to the logical partition even if the 1-4 Partition Number is left.

4. to partition a hard disk, you can modify the partition table of the hard disk. That is to say, you can use the fdisk partition to modify the 64-byte partition table. The data corresponding to the partition does not matter. Therefore, theoretically adjusting the partition size does not delete data in the partition.

Partition Scheme

Solution 1: A node in the cluster structure has multiple copies or is unimportant.

/Boot 100 MB linux boot program

The physical memory of swap is 1.5 times. When the memory is greater than or equal to 8 GB, 8 GB is enough.

/Remaining hard disk size

Method 2: database and storage with a large amount of important data

/Boot 100 MB

/50-200G

The physical memory of swap is 1.5 times. When the memory is greater than or equal to 8 GB, 8 GB is enough.

The remaining hard disk size of/data is stored in the database and stored data.

Solution 3: Portal-level or large websites

/Boot 100 MB

The physical memory of swap is 1.5 times. When the memory is greater than or equal to 8 GB, 8 GB is enough.

/50-200G

The remaining space is retained and no partitions are made. The remaining space will be allocated to other required departments in the future.

Device Name of the hard disk partition

In linux, the device name is used to access the device. The device name is stored in the "/dev" directory.

The device name definition rules are as follows, and other partitions are as follows:

Hard Disk of the first IDE interface of the system:/dev/hda

Hard Disk of the second IDE interface of the system:/dev/hdb

Hard Disk of the first SCSI interface of the system:/dev/sda

Hard Disk of the second SCSI interface of the system:/dev/sdb

Both SATA and SAS start with sd.

Each partition is represented by a disk name and a corresponding number:

The first partition of the system's first IDE interface Hard Disk:/dev/hda1

The fifth partition of the system's first IDE interface Hard Disk:/dev/hda5

The first partition of the second SCSI interface hard disk of the system:/dev/sdb1

The fifth partition of the second SCSI interface hard disk of the system:/dev/sdb5

Note:

1. The Partition Number 1-4 is reserved for the primary partition or extended partition. The logical Partition Number can only start from 5.

2. The names of all IDE Hard Drive devices start with/dev/hd. The hard drive numbers are/dev/hda/,/dev/hdb,/dev/hdc ....

3. The device names of SCSI/SAS/SATA/USB interface hard disks start with/dev/sd. The hard disk numbers are/dev/sda,/dev/sdb,/dev/sdc ....

4. Pay special attention to the fact that SAS/SATA is the mainstream Hard Drive Interface in the Internet production environment, and ssd solid state disks are gradually applied because they are electronic devices with high performance.

5. There is no drive letter concept in linux. To operate on disk devices, you need to use the disk device name to read the disk device content. You need to mount the file system of the created partition or logical volume to the specified directory.

6. After partitioning on linux, you must create a file system on the partition.

File System Type

A file system organizes data and metadata on a storage device. A file system organizes files on a disk (including a CD, floppy disk, flash disk, and other storage devices) or partition, common file systems include ext2, ext3, ext4, NTFS, FAT, and FAT32. A file system is a method used to organize and store files or data. It is designed to facilitate data query and access. Therefore, data cannot be stored without a file system on the disk. Therefore, you must create a file system before using the disk partition.

The file system is based on storage devices, such as hard disks or CDs, and contains maintenance of the physical location of files. The file system can also be said to be a method for storing virtual data or network data, for example, NFS, MFS, and GFS. Currently, unix and linux file systems have many implementations, such as UFS (BSD-based unix File System), ext2, ext3, ext4, zfs, xfs, and reiserfs.

Note:

1. The file system organizes data and metadata on a storage device.

2. partitions must be formatted to create a file system to store data.

3. A partition can only have one file system.

4. Common file systems in linux: ext2, ext3, ext4, zfs, xfs (CentOS7), and reiserfs (separately installed ). NTFS and FAT32 are common file systems in windows.

File System Selection

SAS and SATA silver disk file system selection:

1. reiserfs: reiserfs (within KB) is the preferred choice for businesses with large volumes of small files)

2. xfs: Some portal data block mysql services.

4. ext4: video download, streaming media, database, and small file services.

5. ext2: No logs, CDN website acceleration service.

Common applications:

CentOS5.x: ext3 by default.

CentOS6.x: ext4 by default.

CentOS7.x: xfs by default.

SSD File System Selection

Ext4 and reiserfs can be used as SSD file systems, but they are not optimized. They cannot fully utilize SSD performance and affect SSD time.

Btrfs optimizes SSD, and mount is enabled by parameters. However, btrfs is still in the test phase and the production environment should be used with caution.

Jffs32, nilfs2, and yaffs are commonly used flash file systems. They are widely used in Embedded environments and are recommended for use.

Create a file system in linux

Use a command similar to the following to create and optimize the reiserfs File System in linux:

/Sbin/mkreiserfs/dev/sda2

Use a command similar to the following to create an xfs File System in linux:

Mkfs-t xfs-f/dev/sda2

Note: ext3fs simply activates the ext2fs for billing. When creating the file system, use the-j option for mke2fs:

/Sbin/mke2fs-j/dev/sda2

View currently supported file systems

File System

Super block: records control and management information of the file system (information earlier than Group0 ).

1. number and size of blocks and inode, and the number of used and unused blocks.

2. File System loading time, last Data Writing Time, last fsck disk check time, etc.

3. Valid bits: 0 is loaded, and 1 is loaded.

Group Description: record where the block group starts.

Block map: Use a map to record which blocks are used and which ones are not used.

Inode map: Use a map to record inode usage.

Inode: an area composed of inode. Each inode has a unique ID.

Data block area: a block area used to store file data.

Disk Partition

Add a disk and create 6 partitions

1 p + 1e (5L):/dev/sdb1,/dev/sdb5,/dev/sdb6,/dev/sdb7, dev/sdb8, dev/deb9

Partition: Set a partition table

Format: create a file system

View disk Information

 

 

 

 

 

 

 

Last Partition

List types

 

Notify linux partition table changes (otherwise, you must restart the system to see the changes)

View results

Simple file system operations

Disk and directory capacity:

Df: list the total disk usage of the file system.

Du: Evaluate the disk usage of the file system (usually used to evaluate the directory capacity ).

1. display the capacity results in a readable capacity format: df-h.

2. display the available disk capacity under/etc in a readable capacity format: df-h/etc.

3. List the number of inode available for each partition: df-ih.

4. Check the capacity occupied by each directory under the root directory: du-sm /*.

Connection file: ln

1. hard link: Create a new file name in a directory to link to the association record of an inode number.

2. symbolic link (symbolic link, shortcut): create another independent file, which allows the data to read the file name of the folder linked to it.

3. Copy/etc/passwd to/tmp and view inode and block

Disk format: mkfs-t ext3/dev/sda6

Disk check: fsck and badblocks

Fsck-C-f-t ext3/dev/sda6, uninstall status check.

Badblocks-sv/dev/sda6

Attach and detach a disk

1. Mount ext2 and ext3 file systems

Mount/dev/sda6/mnt

Df-h

2. Attach a cd or dvd.

Mkdir/media/cdrom

Mount-t iso9660/dev/cdrom/media/cdrom #-t iso9660 this is the disc format

Mount/dev/cdrom/media/cdrom # test mounting by the System

Df

3. Format and mount a floppy disk

Mkfs-t vfat/dev/fd0

Mkdir/media/floppy

Mount-t fat/dev/fd0/media/floppy

Df

4. mount a USB flash drive

Mkdir/tmp/flash

Mount-t vfat-o iocharset = cp950/dev/sda1/tmp/flash

Df

5. unmount the umount File

Umount/dev/sda6

Umount/media/cdrom

Umount/tmp/flash

Umount/dev/fd0

Umount/tmp/home

Force unmount: umount-lf mount point

Case of force detach nfs:

Umount:/mnt: device is busy Solution

When the nfs mount directory cannot be detached, the reasons and solutions are as follows:

1. The current directory may be the mounted nfs Directory (/mnt)

Solution: exit the mounted directory/mnt and uninstall it.

2. For example, if the nfs server fails, the nfs client will have a problem (the df-h window will die)

The best method to force uninstall umount-lf/mnt.