Linux reading abstracts--linux disk and file system Management __linux

statement: This article is "Brother Bird's Linux private dish" reading summary.

Disk Basics Dot Finishing

The main components of the whole disk are: a rounded disk disc (a portion of the main record data); a mechanical arm, with a disk-reading head on a robotic arm (the data on a disk that can be erased); a spindle motor that rotates a disk so that the reading head of the mechanical arm reads and writes data on the disk disc.

Sector (sector) is the smallest physical storage unit, and according to the different disk design, there are currently 512bytes and 4K two kinds of formats; the sector is composed of a circle, that is, the magnetic column (Cylinder); The early partition was mainly the magnetic column as the smallest partition unit, Partitions are now typically using sectors as the smallest partition units; There are two main formats for disk partition tables, one is a more restrictive MBR partition table, and one is a newer, less restrictive GPT partition table. In the MBR partition table, the first sector is the most important: (1) The main boot area (master boot record, MBR) and the partition table (partition table), where the MBR occupies 446bytes and partition table occupies 64 bytes GPT Partitioned tables can support more than 2TB of disk capacity in addition to a larger number of partitions.

Linux's Orthodox filesystem for EXT2 (Linux second extended file system, EXT2FS)

A data that can be mounted is a file system instead of a partition slot. The file system of the Linux operating system includes file data, file permissions (rwx) and file attributes (owner, group, time parameter, etc.). The file system typically stores these two parts in separate chunks, with permissions and attributes placed in the inode, and the actual data is placed in the data block blocks. In addition, there is a super block (Superblock) will record the entire file system information, including the total number of inode and block, usage, surplus and so on.

Superblock: Record The overall information of this filesystem, including the total amount of inode/block, usage, surplus, and file system format and related information, etc.
Inode: Records the properties of a file, a file occupies an inode, while recording the file's data in the block number;
Block: The contents of the actual record file, if the file is too large, will occupy multiple blocks.

The reason that fat systems need to be defragmented often is because file writes are too discrete, and the efficiency of file reads is poor. This is the time to defragment the blocks that belong to the same file, so that the data can be read easily.

the Linux EXT2 file System file system starts with the inode and block planning, unless reformatting (or using RESIZE2FS and other instructions to change the file system size), otherwise the inode and block are fixed with no change. But the file system up to hundreds of G put all the inode and block together a large number of difficult to manage. Therefore, the Ext2 file system is basically divided into multiple block groups (blocks group) in the format, each block group has a separate inode/block/superblock system.

Boot Sector

In overall planning, there is a boot sector (boot sector) at the top of the filesystem, and this boot sector can be installed with the boot manager, which is a very important design, because we are able to install different boot management programs to the front of the individual file system. Instead of covering the entire disk's unique MBR, this can create a multiple-boot environment.

Data block (data blocks)
The data block is used to place the file content data, which is supported in the Ext2 file system by 1 K, 2K and 4K three species. The block size is fixed when it is formatted, and each block is numbered to facilitate the logging of the inode.     However, it is important to note that the maximum disk capacity that the file system can support is not the same as the maximum single file capacity due to differences in block size. In principle, the size and number of blocks can no longer be changed after formatting (unless reformatted); a maximum of one file can be placed in each block, and if the file is larger than the size of blocks, then a file will occupy more than one number. If the file is smaller , the remaining capacity of the block can no longer be used (disk space is wasted).

Inode table (inode form)
The file data recorded by the inode includes at least: the access mode (Read/write/excute) of the file, the owner and Group (Owner/group) of the file, the capacity of the file, the time when the file was created or changed (CTime), and the last Read time ( Atime); Recently modified time (Mtime), flag (flag) defining file attributes, such as SetUID ... The true contents of the document refer to the box (pointer);
The number and size of inode are also fixed when formatted, in addition to other features: Each inode size is fixed to 128 bytes (new EXT4 and XFS can be set to 256 bytes); Each file occupies only one inode; The number of documents that can be established by the EC is related to the number of inode; When the system reads the file, it needs to find the inode and analyze whether the permissions recorded by the Inode are in line with the user, and the content of the block can begin to be read.
Inode to record a lot of data, but only 128bytes only, and the Inode record a block number to spend 4byte, if I have a file with 400MB and each blocks for 4K, then at least 100,000 of the record of a block number 。 The inode has so much to record information. For this reason our system is very clever to define the area of the Inode record block number as 12 direct, one indirect, one double indirect and one three indirect record area. as shown below. Based on the above information can be calculated when block is 1k, the largest file for 2G origin. However, you cannot use this to calculate blocks of 2k and 4k.

superblock
    Superblock is where information about the entire filesystem is recorded, without superblock, there is no this It's filesystem. He recorded the main information: block and Inode total, unused and used Inode/block quantity, block and inode size (Block 1, 2, 4 K, inode is 128bytes or 256bytes); fi Lesystem time of the mount, information about the last time the data was written, the time of the most recent test disk (FSCK), and other file systems; a valid bit value, if the filesystem has been mounted, the valid bit is 0, and if not mounted, the valid bit is 1.
    Superblock is important because the basic information of our filesystem is written here, so if Superblock dies, your filesystem may take a lot of time to save. Generally speaking, the size of the superblock is 1024bytes. Moreover,. Each block group may contain superblock in fact, in addition to the first block group will contain Superblock, the subsequent block group does not necessarily contain superblock, and if it contains superblock the Superblock is mainly as the first block group within the Superblock backup slightly, so that can carry out superblock rescue.

filesystem Description (File system description)
    This section can describe the start and end block numbers for each block group. and a description of which block number each section (Superblock,bitmap,inodemap,data block) is located between. This part can also be observed with DUMPE2FS.

block Bitmap (chunk comparison)

From the block bitmap you know which blocks are empty, so our system can quickly find the available space to dispose of the files. Similarly, if you delete certain files, then the block number that the file originally occupies must be released, at this time the tag corresponding to the block number in the block bitmap must be modified to be "unused." This is the function of bitmap.

Inode Bitmap (inode comparison table)
This is actually similar to block bitmap, except that the block bitmap record is used with unused block numbers, as for Inode bitmap is recorded using an unused inode number.

dumpe2fs: Query ext Family superblock Information Instructions (note cannot be used to search for XFS file system)

Directory

The contents of the directory in the record file name, the general file is the actual record of data content. When we create a directory on the Linux file system, the file system assigns an inode and at least one block to the directory. Where the Inode records the relevant permissions and attributes of the directory, and records the block code assigned to it, and the blocks are records of the file name in this directory and the Inode number data that the file name occupies. That is, the block content that the directory occupies is recorded in the following information:

If you want to actually look at the inode number that the file in the root directory occupies, you can use the LS-I option to handle:

File
When we build a generic file under Linux ext2, Ext2 assigns an inode to the file with a block number relative to the size of the file. For example, suppose that one of my blocks is 4 Kbytes and I want to build a Kbytes file, then Linux allocates an inode with 25 blocks to store the file. Note, however, that because the inode has only 12 direct points, there is one more block to record as a chunk number.

The directory tree reads The inode itself and does not record the file name, which is in the block of the directory. Therefore, in the fifth chapter of the document and Directory permission statement, we will mention the "Add/remove/rename file name and directory of W Permissions" features. So because the file name is recorded in the directory block, so when we want to read a file, it is necessary to go through the directory of the Inode and blocks, and then can find the file to be read the inode number, the final will read the correct file of the data inside.
If I want to read/etc/passwd This file, how does the system read it?
1./'s Inode:
The inode number 128 is found through the information of the mount point, and the permission of the Inode specification allows us to read the contents of the block (with R and X);
2./'s BLOCK:
After the last step, obtain the block number and find the inode number (33595521) with the etc/directory;
3.etc/'s Inode:
Read the No. 33595521 Inode learned that Dmtsai has R and X permissions, so you can read etc/block content;
4.ETC/'s BLOCK:
After the last step, get the block number and find the inode number (36628004) with the passwd file in the content;
Inode for 5.PASSWD:
Read the No. 36628004 Inode learned that Dmtsai has R permission, so can read passwd block content;
Block of 6.PASSWD:
Finally, the data of the block content is read out.

How does the system read if I want to create a new file or directory?

1. First determine whether the user wants to add files to the directory has W and x permissions, if any can be added;
2. According to Inode bitmap find unused inode number, and the new file permissions/attributes written;
3. According to block bitmap find the block number not in use, and the actual data into the blocks, and update the inode of the blocks point to data;
4. Synchronize the Inode bitmap with block bitmap and update the contents of C Superblock with the data just written.

In order to prevent the result of inconsistent state of data, the file check is too slow, the log file system rise

In filesystem, a block is planned, which is designed to record the steps of writing or revising a file, and that does not simplify the process of consistency checking. Other words:
1. Preparation: When the system to write a file, it will first record in the log block of a file to write information;
2. Actual writing: Permission and data to begin writing files; start updating metadata data;
3. Finish: After completing the data and metadata update, complete the document record in the log block.
Ext2 can achieve this function: through the EXT3/EXT4 can be. EXT3/EXT4 is an upgraded version of Ext2 and can be compatible with the ext2 version.

The relationship between file systems and memory on a Linux system

The system will put the common file data into the main memory buffer to speed up the file system read/write;
, so that the physical memory of Linux will be exhausted in the end. This is the normal situation. Can accelerate the system efficiency;
You can manually use sync to force the files in memory set to Dirty to be written back to disk;
If the normal shutdown, shutdown instructions will actively call sync to the memory of the data back to the disk;
However, if the shutdown is not normal (such as a skip, a machine, or other unknown reason), because the data has not been written back to the disk, the reboot may spend a lot of time on disk testing, and may even cause file system damage (not disk damage).

significance of Mount point

The action of combining file system with directory tree is called "Mount". A mount point must be a directory, which is the entry into the filesystem. So not that you have any file system to use, you have to "mount" to a directory in the directory tree before you can use the file system's

Linux-supported file systems and VFS

Traditional file system: Ext2/minix/ms-dos/fat (with VFAT module)/iso9660 (CD-ROM) and so on;
Log file system: Ext3/ext4/reiserfs/windows ' NTFS/IBM ' s Jfs/sgi ' s XFS/ZFS
Network File system: NFS/SMBFS

The entire Linux system is read filesystem through a core function called Virtual filesystem switch. In other words, the whole Linux know filesystem is actually the VFS management, we users do not need to know what each partition filesystem is what ~VFS will actively help us do the reading action.

Introduction to XFS File System

Ext family Current More nerve-racking place: support is the most extensive, but the format is very slowconfiguration of XFS file systemXFS file system in the distribution of data, the main planning for three parts, a data section, a file system active login area (log section) and a real-time operation area (Realtime section). The data for these three regions are as follows:
profile (data section)
Includes data such as Inode/data Block/superblock. The data that is required to place the file system separately is divided into multiple storage groups (allocation groups). Each storage area group includes (1) the superblock of the entire file system, (2) the management mechanism of the remaining space, and (3) The allocation and tracing of the inode. In addition, the inode and block are the system needs to use, this dynamic configuration generated, so the format of the action super fast. Unlike the Ext family, the XFS block and Inode have a number of different capacity to set, block capacity can be provisioned by 512bytes ~ 64K, however, in the Linux environment, due to the memory control of the relationship (paging file pagesize capacity), Therefore, the maximum block size that can be used is 4K. As for the inode capacity can be from 256bytes to 2M so large. However, it is probably enough to keep the default value of 256bytes.

File System active Login area (log section)
Login area This area is mainly used to record file system changes, in fact a bit like the log area.
Real-time Operation area (Realtime section)
When a file is to be built, XFS will find a number of extent blocks in this section, place the file in the block, wait until the assignment is complete, and then write to the Inode and block in data sections. The size of this extent block must be specified at the time of format, with a minimum value of 4K to 1G.

Xfs_info to observe the instructions of the Superblock content

The isize in line 1th refers to the capacity of the inode, each 256bytes so large. As for Agcount, the number of storage group groups (allocation group) mentioned earlier, a total of 4, agsize means that each storage group has 65,536 block.     With block 4th set to 4K, so the entire filesystem capacity should be 4*65536*4k so large.     In line 2nd, Sectsz refers to the logical sector (sector) capacity set to 512bytes so large meaning. The bsize in line 4th refers to the block capacity, which means that each block is 4K, with a total of 262,144 blocks in the file system.
The Sunit and Swidth in line 5th have a higher correlation with the stripe of the disk array. We'll give you an example when we format this part.
The internal in line 7th refers to the location of the login area within the file system, not the external device meaning. and occupies 4K * 2,560 blocks, a total of about 10M capacity.
The 9th line inside the realtime area, inside the extent capacity of 4K. But it is not currently in use.

File System Operations directory and capacity of diskDF: Lists the overall disk usage of the file system;
Du: Evaluate the disk usage of the file system (commonly used in the capacity of the estimate directory)

Entity links and Symbolic Links: ln      There are two types of links under Linux, one that is similar to Windows shortcuts, allowing you to quickly link to target files (or directories); The other is through the file system's Inode link to generate a new filename, rather than create a new file. This is called an entity link (hard links).

Hard Link (entity links, hard links, or actual links)
    Each file occupies an inode, and the contents of the file are pointed to by the record of the inode; to read the file, you must The filename of the directory record to point to the correct inode number to read. Hard link simply adds a file name to a directory that links to an inode number.