Linux file system soft link hard link

Intro

There are many implementations of UNIX file systems, such as UFS (BSD-based UNIX file systems), Ext3, Ext4, ZFS, ReiserFS, and so on.

Regardless of the file system, you always need to store the data. the minimum storage unit for a hard disk is a sector , and the smallest unit of data storage is not a sector, because the storage efficiency of the sector is too low. A sector has only 512 bytes, and the head is a sector of a sector to read, that is, if the file is 10MB, then in order to read this file, the head must be read 20,480 times. Such efficiency is extremely low.

Logical Block

In order to improve efficiency, there is the concept of block, or it can be called a data block. The logical block is the "minimum storage unit" specified when the partition is formatted with the file system, and the minimum storage unit is sector-based, so the size of the logical block is always 2 times the N times of the sector. At this point, the head can read one block at a time, so the efficiency is high!

Logical block planning is very learned, not the bigger the better, because a logical block can only hold a file (in Linux ext2), so if the logical block is planned too large, then it will be a waste of disk space. For example, if a logical block is 4KB, and a file has only 0.1KB size, and the small file still occupies a logical block, it will waste 3.9KB of space.

Therefore, you need to consider the purpose of the host when planning your disk. such as BBS host, because the article short, file smaller, then the logical block allocation of a little bit better. If the host is mainly used to store large volumes of files, then consider the use of efficiency, or logical block a little bit better!

the composition of the disk

We can divide a disk into one or more partitions. Each partition can contain a single file system.

Let's describe a hierarchical refinement process below, and ask you to focus on what I think:

1 disks are made up of one partition, that is, disk = partition + partition + partition ...

2 There is a file system within each partition, and there is only one file system within a partition.

3 Each partition contains these contents in sequence: Bootstrap block (also called Bootstrap Block), super block, cylinder group 0, cylinder Group 1, ... Cylinder Group N. That is, partition = Bootstrap block + Super Block + Cylinder Group (several)

4 Each cylinder group also includes these contents: Super Block Copy, configuration information, I node graph (record which I nodes are available), block bitmap (to record which blocks are available), I node (many), data block (also called logical block)

Well, you should be able to construct a hierarchical map in your mind based on 1,2,3,4, and if you draw it out, it will be more beneficial to the concept of memory I node.

Super Block

The function of the Super Block (Superblock) is to store the size of the file system, empty and filled blocks, and their respective totals and other such information. To use a partition for data access, the first one to access is the Super block. So, if the super block is broken, the disk is basically hopeless.

I node

The following is about the I node, you can not mention the security of Linux. Because the Linux operating system is a multi-user, multi-tasking environment, in order to protect the privacy of each user's own data, each file is divided into two parts to store: One is the properties of the file, the other is the contents of the file.

The I node (iinode) is used to store the properties of a file, whereas a block of data (logical block) is used to store the contents of a file!

If you want to format a partition, specify the size of the inode and the size of the block! More generally, a ext2 file system is sure to include the Inode table with the block area of these two parts!

As for the block, as I mentioned earlier, it is also called a logical block, also called a data block, which is the smallest unit of data storage.

And the Inode " record file attributes and where the contents of the file is placed in " the information, more generally, the inode in addition to the attributes containing the file, but also includes a pointer, which points to the location of the file contents of the data block placed, So that the operating system can easily read the contents of the file.

Such file attribute information is generally included in the inode:

• The owner of the file and the group of users to whom it belongs;
• The access permission setting of the file;
• The type of file;
• Time of file access, modification, etc.
• The size of the file;
• Documents of various flags, such as SUID and sgid, etc.;
• A pointer to the file content data block.

The size of an inode is typically 128 bytes. (This knowledge will be subverted in Ext4, and the inode size in EXT4 will be expanded to 256 bytes)

Well, let's see how we can manage files using Inode.

Catalogue

Let's take a look at the details of the directory operation:

If we set up a directory, then the system assigns an inode and at least one block to the directory. The Inode records the relevant properties of the directory and points the pointer to the allocated data block. In the allocated block, the correlation of the related files (and subdirectories) in this directory is recorded, and more generally, a table containing three columns is stored in the directory block, and the three columns are: inode, file name or directory name, pointer to the data block.

We use the VI command to see what the contents of a directory are: (of course, this is just what the user sees, and the underlying implementation of the filesystem is different.) ）

" ============================================================================
" Netrw Directory Listing                                        (netrw v109)
"   /rocrocket/PSB/home/git27
"   Sorted by      name
"   Sort sequence: [\/]$,\.h$,\.c$,\.cpp$,\.[a-np-z]$,*,\.info$,\.swp$,\.o$\.obj$,\.bak$
"   Quick Help: <F1>:help  -:go up dir  D:delete  R:rename  s:sort-by  x:exec
" ============================================================================
../
./
.git/
roc.c

The double quotation marks begin with the comment section followed by four items, the first two being the parent directory and the current directory that are intrinsic to any directory, and then a hidden directory. Git, and finally a file roc.c in the current directory. It can be seen that a directory is actually a file, except that it does not store user data, but instead stores the list of files and subdirectories in the directory.

If you create a new normal file in Linux, the file is assigned at least one inode and the number of blocks relative to that file size. For example, if a block is 4KB and a 100KB file is built, Linux allocates an inode with 25 blocks to store the file.

One thing to be special! Special! Special! The reminder is that the inode itself does not record the file name, but rather the related attributes of the document (those mentioned above), and the file name is recorded in the block area to which the directory belongs. For this reason, if Linux reads the contents of a file, it is necessary to get the inode of the file by the root directory/the inode where the file's upper-level directory is located, and then the file association recorded by that directory. Finally, the final file content is obtained through the block pointers provided within the inode.

Link Count

And when it comes to the number of links, here are some concepts and some regular conclusions:

There is a link count in each I node whose value is the number of directory entries that point to the I node.

The file can be deleted only if the link technology is reduced to 0 o'clock (that is, releasing the data blocks that the file occupies)

Links that can increase the number of links are hard links .

A soft link is also called a symbolic link, and its inode file type is S_iflnk. It just stores the path and name of the other file.

The link count for any one leaf directory (a directory that does not contain any other directories) is always 2, and the value 2 comes from the directory entry that names the directory and the. Item in that directory.

Each subdirectory in the parent directory causes the link count of the parent directory to be increased by 1.

Wonderful citation

Finally gives the CSDN online qxp Netizen's section about the soft link and the hard link comment, very good:

We know that UNIX files can be broadly divided into three parts: directory (file name), Inode, and data area.

For replication, not only the new directory entry (file name), the new inode, but also all the data of the file are copied;

The hard link only creates a new catalog entry , and the corresponding inode number in the catalog entry is linked to the inode number of the corresponding file, and the Inode reference count of the file is added 1;

This way, when you delete the original file, the file data is not deleted because the Inode node reference count is >0, so you can continue to access it through hard links.

In other words, a hard connection causes the file to have another alias, another entry.

By the way, the soft link, the symbolic link, is actually the equivalent of a shortcut under Windows.

Creates a new directory entry, a new inode, except that the referenced file path and name are placed in the data area.

Links and differences between hard links and soft links

We know that files have filenames and data, which are divided into two parts on Linux: User data and metadata (metadata). User data, which is the file block (data block), is where the real content of the file is recorded, while metadata is an attached property of the file, such as file size, creation time, owner, and so on. In Linux, the inode number in the metadata (the Inode is part of the file metadata but does not contain a file name, the inode number is the index node number) is the unique identifier of the file and not the filename. The file name is only for the convenience of people's memory and use, the system or program through the inode number to find the correct file data block. Figure 1 shows the process by which a program obtains the contents of a file by its filename.

Figure 1. Open File by file name

Listing 3. Move or rename a file

 # stat /home/harris/source/glibc-2.16.0.tar.xz 
  File: `/home/harris/source/glibc-2.16.0.tar.xz‘
  Size: 9990512   	 Blocks: 19520      IO Block: 4096   regular file 
 Device: 807h/2055d 	 Inode: 2485677     Links: 1 
 Access: (0600/-rw-------)  Uid: ( 1000/  harris)   Gid: ( 1000/  harris) 
 ... 
 ... 
 # mv /home/harris/source/glibc-2.16.0.tar.xz /home/harris/Desktop/glibc.tar.xz 
 # ls -i -F /home/harris/Desktop/glibc.tar.xz 
 2485677 /home/harris/Desktop/glibc.tar.xz

To view the inode number in a Linux system, you can use command stat or ls-i ( use command Istat if AIX system). Listing 3. Use the command MV to move and rename the file glibc-2.16.0.tar.xz, the result does not affect the file user data and inode number , the file before and after the migration of the inode number is: 2485677.

To solve the shared use of files, the Linux system introduces two links: Hard link and soft link (also known as symbolic link, soft link or symbolic link). Link for Linux system to solve the shared use of files, but also bring hidden file paths, increase permissions security and save storage and other good places. If an inode number corresponds to more than one file name, the files are called hard links. In other words, a hard link is the same file using multiple aliases (see Figure 2.hard, Link is an alias of file, they have a common inode). hard links can be created by the command link or ln . as follows, create a hard link to the file oldfile.

 link oldfile newfile 
 ln oldfile newfile

because a hard link is a file with the same inode number with only a different file name , hard links have the following characteristics:

• The file has the same inode and data block;
• Only files that already exist can be created;
• Cannot cross file system for hard link creation;
• The directory cannot be created, only the file can be created;
• Deleting a hard-link file does not affect other files that have the same inode number.

Listing 4. Hard link feature display

# ls -li
 total 0

 // Only hard links can be created for existing files
 # link old.file hard.link
 link: cannot create link `hard.link‘ to `old.file’: No such file or directory

 # echo "This is an original file"> old.file
 # cat old.file
 This is an original file
 # stat old.file
  File: `old.file‘
  Size: 25 Blocks: 8 IO Block: 4096 regular file
 Device: 807h / 2055d Inode: 660650 Links: 2
 Access: (0644 / -rw-r--r--) Uid: (0 / root) Gid: (0 / root)
 ...
 // files have the same inode number and data block
 # link old.file hard.link | ls -li
 total 8
 660650 -rw-r--r-- 2 root root 25 Sep 1 17:44 hard.link
 660650 -rw-r--r-- 2 root root 25 Sep 1 17:44 old.file

 // cannot cross file system
 # ln / dev / input / event5 /root/bfile.txt
 ln: failed to create hard link `/root/bfile.txt‘ => `/ dev / input / event5’:
 Invalid cross-device link

 // Cannot create a hard link to the directory
 # mkdir -p old.dir / test
 # ln old.dir / hardlink.dir
 ln: `old.dir /‘: hard link not allowed for directory
 # ls -iF
 660650 hard.link 657948 old.dir / 660650 old.file

File Old.file and Hard.link have the same inode number: 660650 and file permissions, the Inode is present as the file exists, so you can create a hard link only if the file exists, that is, when the inode exists and the link count is not 0 o'clock 。 The inode number is unique only under each filesystem, and when Linux mounts multiple file systems, the inode repetition occurs (as shown in Listing 5. File t3.jpg, Sync, and 123.txt are not associated, but have the same inode number), so hard-link creation cannot span files System. The file system used by the device files directory/dev is Devtmpfs, and/root (with root/consistent) uses the disk file system Ext4. Listing 5 shows the use of the command DF to view the file system types mounted on the current system, the file system inode usage, and the file system mount points.

Listing 5. Find files with the same inode number

 # df -i --print-type 
 Filesystem     Type       Inodes  IUsed    IFree IUse% Mounted on 
 /dev/sda7      ext4      3147760 283483  2864277   10% / 
 udev           devtmpfs   496088    553   495535    1% /dev 
 tmpfs          tmpfs      499006    491   498515    1% /run 
 none           tmpfs      499006      3   499003    1% /run/lock 
 none           tmpfs      499006     15   498991    1% /run/shm 
 /dev/sda6      fuseblk  74383900   4786 74379114    1% /media/DiskE 
 /dev/sda8      fuseblk  29524592  19939 29504653    1% /media/DiskF 

 # find / -inum 1114 
 /media/DiskE/Pictures/t3.jpg 
 /media/DiskF/123.txt 
 /bin/sync

It is worth mentioning that the Linux system has the inode number is exhausted but the disk space still has the remaining situation. We create a 5M size EXT4 type of mo.img file and mount it to the directory/mnt. Then we use a shell script that will mount the Indoe depletion of the Ext4 file system under/MNT (see Listing 6.).

Listing 6. Test a scenario where the file system Inode is exhausted but still has disk space

 # dd if=/dev/zero of=mo.img bs=5120k count=1 
 # ls -lh mo.img 
 -rw-r--r-- 1 root root 5.0M Sep  1 17:54 mo.img 
 # mkfs -t ext4  -F ./mo.img 
 ... 
 OS type: Linux 
 Block size=1024 (log=0) 
 Fragment size=1024 (log=0) 
 Stride=0 blocks, Stripe width=0 blocks 
 1280 inodes, 5120 blocks 
 256 blocks (5.00%) reserved for the super user 
 ... 
 ... 
 Writing superblocks and filesystem accounting information: done 

 # mount -o loop ./mo.img /mnt 
 # cat /mnt/inode_test.sh 
 #!/bin/bash 

 for ((i = 1; ; i++)) 
 do 
    if [ $? -eq 0 ]; then 
        echo  "This is file_$i" > file_$i 
    else 
        exit 0 
    fi 
 done 

 # ./inode_test.sh 
 ./inode_test.sh: line 6: file_1269: No space left on device 

 # df -iT /mnt/; du -sh /mnt/ 
 Filesystem     Type Inodes IUsed IFree IUse% Mounted on 
 /dev/loop0     ext4   1280  1280     0  100% /mnt 
 1.3M 	 /mnt/

hard links cannot be created on a directory that is constrained by the design of the file system (see Listing 4. Creating a hard link to a directory will fail). Directories in the Linux file system now have two special directories hidden: the current directory (.) and the parent directory (.. ）。 Looking at the inode numbers for these two special directories is the fact that these two directories are two hard links (note the inode number of the directory/mnt/lost+found/). If the system allows hard links to be created on the directory, a directory ring is generated.

 # ls -aliF /mnt/lost+found 
 total 44 
 11 drwx------ 2 root root 12288 Sep  1 17:54 ./ 
 2 drwxr-xr-x 3 root root 31744 Sep  1 17:57 ../ 

 # stat  /mnt/lost+found/ 
  File: `/mnt/lost+found/‘
  Size: 12288     	 Blocks: 24         IO Block: 1024   directory 
 Device: 700h/1792d 	 Inode: 11          Links: 2 
 Access: (0700/drwx------)  Uid: (    0/    root)   Gid: (    0/    root) 
 Access: 2012-09-01 17:57:17.000000000 +0800 
 Modify: 2012-09-01 17:54:49.000000000 +0800 
 Change: 2012-09-01 17:54:49.000000000 +0800 
 Birth: -

A soft link differs from a hard link in that the file is a soft connection if the contents of the file's user data block are pointing to the path name of another file . Soft link is a normal file, but the content of the data block is a bit special. The soft link has its own inode number and user data block (see Figure 2.). Therefore, the creation and use of soft links does not have many restrictions similar to hard links: soft connections can only be created with the Ln-s command and cannot be created using link

• Soft links have their own file attributes and permissions, etc.;
• You can create a soft link to a nonexistent file or directory;
• Soft link can cross file system;
• Soft links can be created on files or directories;
• When you create a soft link, the link count i_nlink not increase;
• Deleting a soft link does not affect the file being pointed to, but if the original file being pointed to is deleted, the associated soft connection is called a dead link (that is, dangling link, if it is re-created by pointing to the path file, the dead link can revert to the normal soft link).

Figure 2. Access to Soft links

Listing 7. Soft link Feature Display

# ls -li
  total 0

  // Can create soft links to non-existing files
  # ln <span style = "color: # FF0000;">-s </ span> old.file soft.link
  # ls -liF
  total 0
  789467 lrwxrwxrwx 1 root root 8 Sep 1 18:00 soft.link-> old.file

  // Because the pointed file does not exist, the soft link at this time soft.link is a dead link
  # cat soft.link
  cat: soft.link: No such file or directory

  // Create the pointed file old.file, and soft.link reverts to a normal soft link
  # echo "This is an original file_A" >> old.file
  # cat soft.link
  This is an original file_A

  // Create a soft link to a directory that does not exist
  # ln -s old.dir soft.link.dir
  # mkdir -p old.dir / test
  # tree. -F --inodes
  .
├── [789497] old.dir /
│ └── [789498] test /
├── [789495] old.file
├── [789495] soft.link-> old.file
└── [789497] soft.link.dir-> old.dir /

Of course, the user data of soft link can also be the path of another soft link, whose parsing process is recursive. However, it is important to note that the path of the original file when the soft link is created is better with absolute path. When a soft link created with a relative path is moved, the soft link file becomes a dead link (the soft link A as shown below uses a relative path and therefore should not be moved) because the linked data block also points to a relative path.

 $ ls -li 
 total 2136 
 656627 lrwxrwxrwx 1 harris harris       8 Sep  1 14:37 a -> data.txt
 656662 lrwxrwxrwx 1 harris harris       1 Sep  1 14:37 b -> a 
 656228 -rw------- 1 harris harris 2186738 Sep  1 14:37 data.txt 6

Link Related commands

In Linux, view the file system types that are currently hanging on the system, in addition to the command DF used above, you can also use the command mount or view the file/proc/mounts.

# mount 
 /dev/sda7 on / type ext4 (rw,errors=remount-ro) 
 proc on /proc type proc (rw,noexec,nosuid,nodev) 
 sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) 
 ... 
 ... 
 none on /run/shm type tmpfs (rw,nosuid,nodev)

The command LS or stat helps us differentiate between soft links and other files and view file Inode numbers, but a better way to use the Find command is not only to find soft links to a file, but also to find all the hard links for the same inode . (see Listing 8.)

Listing 8. Use the command find to find soft links with hard links

// Find a soft link to the file data.txt in the path / home
  # find / home -lname data.txt
  / home / harris / debug / test2 / a

  // see all hard links in the path / home with the same inode
  # find / home -samefile /home/harris/debug/test3/old.file
  /home/harris/debug/test3/hard.link
  /home/harris/debug/test3/old.file

  # find / home -inum 660650
  /home/harris/debug/test3/hard.link
  /home/harris/debug/test3/old.file

  // List all soft link files under the path / home / harris / debug /
  # find / home / harris / debug / -type l -ls
  656662 0 lrwxrwxrwx 1 harris harris 1 Sep 1 14:37 / home / harris / debug / test2 / b-> a
  656627 0 lrwxrwxrwx 1 harris harris 8 Sep 1 14:37 / home / harris / debug / test2 / a->
  data.txt
  789467 0 lrwxrwxrwx 1 root root 8 Sep 1 18:00 /home/harris/debug/test/soft.link->
  old.file
  789496 0 lrwxrwxrwx 1 root root 7 Sep 1 18:01
  /home/harris/debug/test/soft.link.dir-> old.dir

The system defaults to the value of the inode based on the size of the disk (see Listing 9), which can be modified before the format file system, if necessary. If you type a commandmkfs -t ext4 -I 512/dev/sda4，The disk device / dev / sda4 will be formatted as an ext4 file system with an inode size of 512 bytes.



Listing 9. Viewing system inode values

// View the inode value on disk partition / dev / sda7
  # dumpe2fs -h / dev / sda7 | grep "Inode size"
  dumpe2fs 1.42 (29-Nov-2011)
  Inode size: 256

  # tune2fs -l / dev / sda7 | grep "Inode size"
  Inode size: 256

Linux VFS file system classification


Linux has an extremely rich file system, which can be roughly divided into the following categories:



Network file systems, such as nfs, cifs, etc .;
Disk file system, such as ext4, ext3, etc .;
Special file systems, such as proc, sysfs, ramfs, tmpfs, and so on.


The basis for realizing these file systems and coexisting under Linux is Linux VFS (Virtual File System, also called Virtual Filesystem Switch), which is a virtual file system. As a general file system, VFS abstracts the four basic concepts of the file system: files, directory entries (dentry), inodes, and mount points. It provides the user space layer file system related Interface (see Figure 3. VFS in the Linux system architecture). VFS implements system tuning such as open (), read (), and enables user-space programs such as cp to cross file systems. VFS truly implements the above: everything except files is a file in Linux.



Figure 3. VFS architecture in the system

There are four basic objects for the Linux VFS: the Super Block Objects (Superblock object), the Index node objects (Inode object), the Catalog item object (Dentry object), and the file object (the Files objects). The Super Block object represents an installed file system, an index node object represents a file, and a Catalog item object represents a directory entry, such as a device file event5 in Path/DEV/INPUT/EVENT5, which has four directory item objects:/, dev/, input/, and EVENT5. A file object represents a file that is opened by a process. The relationship between these four objects and the process and disk files is 4. Shown, where D_inode is a hard link. For fast parsing of file paths, Linux VFS Designs Directory Item caches (directory Entry cache, or Dcache).

Figure 4. Processing between the objects of the VFS

Inode in a Linux file system

In Linux, the index node structure exists in the system memory and disk, which can be distinguished into the inode of the VFS inode and the actual file system. The VFS Inode, as an abstraction of the inode in the actual file system, defines the structure Inode and its associated operational inode_operations (see Kernel source include/linux/fs.h).

Listing 10. The Inode and inode_operations structure in VFS

struct inode {
     ...
     const struct inode_operations * i_op; // inode operation
     unsigned long i_ino; // inode number
     atomic_t i_count; // reference counter
     unsigned int i_nlink; // number of hard links
     ...
  }

  struct inode_operations {
     ...
     int (* create) (struct inode *, struct dentry *, int, struct nameidata *);
     int (* link) (struct dentry *, struct inode *, struct dentry *);
     int (* unlink) (struct inode *, struct dentry *);
     int (* symlink) (struct inode *, struct dentry *, const char *);
     int (* mkdir) (struct inode *, struct dentry *, int);
     int (* rmdir) (struct inode *, struct dentry *);
     ...
  }

As listing 10. See, there are two counters per file: I_count and I_nlink, that is, reference count and hard link count. The i_count in the struct inode is used to track the number of files accessed, while I_nlink is the number of hard links to the file seen using commands such as ls-l. Or I_count trace the file in memory, and I_nlink is the disk counter. When the file is deleted, the I_nlink is set to 0 first. These two counters of the file make the Linux system upgrade or program update easy. The system or program can not be closed (that is, the file I_count is not 0), the new file is replaced with the same file name, the new file has its own inode and data block, the old file will be completely deleted after the relevant process is closed.

Listing 11. Inode in File system Ext4

struct ext4_inode {
     ...
     __le32 i_atime; // Last access time of file content
     __le32 i_ctime; // inode modification time
     __le32 i_mtime; // Last modification time of file content
     __le16 i_links_count; // hard link count
     __le32 i_blocks_lo; // Block count
     __le32 i_block [EXT4_N_BLOCKS]; // point to a specific block
     ...
  };

Listing 11. Shows the definition of inode in file system EXT4 (see kernel source fs/ext4/ext4.h). One of the three time definitions can correspond with the command stat to view up to three times. I_links_count is not only used for hard-link counts of files, but also for the number of subdirectories of a directory (the directory does not show the number of hard links, and the command Ls-ld sees the number of subdirectories). Because the file system ext3 has a limit of i_links_count, the maximum number is: 32000 (the limit is canceled in Ext4). Try to verify that the maximum number of directory subdirectories and normal file hard links on the Ext3 file system is visible in Listing 12. Error message. Therefore, there is a difference between the inode of the actual file system and the VFS inode.

Listing 12. Limitations of I_links_count in file system ext3

 # ./dirtest.sh 
 mkdir: cannot create directory `dir_31999‘: Too many links 

 # ./linkcount.sh 
 ln: failed to create hard link to `old.file‘: Too many links

Turn http://roclinux.cn/?p=754

http://www.ibm.com/developerworks/cn/linux/l-cn-hardandsymb-links/

http://blog.csdn.net/u012317833/article/details/14057971

Linux file system soft link hard link