Difference between du and DF and solutions for large gaps

Check disk usage in Linux. df and du are the most common ones, but sometimes the results of DU and DF are different. Sometimes there is a big gap.

1. Explanation of the two Commands　

Du -- disk usage

DF -- Disk Free

2. How du and DF work

2.1 du Working Principle

The du command calls the fstat system one by one for statistical files to obtain the file size. Its data is obtained based on files and has great flexibility. Not necessarily

For a single partition, you can operate across multiple partitions. If the number of files in the target directory is large, the speed of DU will be slow.

2.2 working principle of DF

DF uses the statfs system call to directly read the Shard's super block information to obtain the shard usage. Its data is based on the partition metadata, so it can only target the entire partition.

Because DF reads Super blocks, the running speed is not affected by the number of files.

3. Simulation of inconsistencies between du and DF

The common inconsistency between DF and Du is the problem of file deletion. After a file is deleted, it does not exist in the file system directory, so du will not count the file. However, if

At this time, the running program holds the handle of the deleted file, so the file will not be deleted in the disk, and the information in the super block of the partition will not be changed.

DF also counts the deleted files.

(1) usage of the current root Partition

[[email protected] ~]# df -h /Filesystem            Size  Used Avail Use% Mounted on/dev/mapper/VolGroup00-LogVol00      28G   14G   12G  54% /

(2) create a 1 GB File

[[email protected] ~]# dd if=/dev/zero of=test.log bs=1024k count=10241024+0 records in1024+0 records out1073741824 bytes (1.1 GB) copied, 14.2389 seconds, 75.4 MB/s

(3) Root partition usage

DF Query Result

[[email protected] ~]# df -hFilesystem            Size  Used Avail Use% Mounted on/dev/mapper/VolGroup00-LogVol00     28G   15G   11G  58% //dev/sda1              99M   13M   82M  14% /boottmpfs                1006M     0 1006M   0% /dev/shm

Du Query Result

[[email protected] ~]# pwd/root[[email protected] ~]# du -sh1.1G    .

In this case, the query results are basically the same.

(4) simulate a process to open the file and then delete the file.

[[email protected] ~]# tail -f test.log &[1] 2880[[email protected] ~]# [[email protected] ~]# rm -fr test.log 

(5) Compare the results of DU and DF again

First, check that a process holds the test. Log handle.

[[Email protected] ~] # Lsof | grep test. logtail 2880 root 3R Reg 253,0 1073741824 2254399/root/test. Log(Deleted) # note that the file has been deleted.

[[email protected] ~]# df -hFilesystem            Size  Used Avail Use% Mounted on/dev/mapper/VolGroup00-LogVol00                       28G   15G   11G  58% //dev/sda1              99M   13M   82M  14% /boottmpfs                1006M     0 1006M   0% /dev/shm[[email protected] ~]# du -sh888K    .

We can see that DF has not changed, while du does not count the deleted file.

(6) Stop the simulation process and compare the results of DU and DF again.

[[email protected] ~]# kill -9  2880[[email protected] ~]# [1]+  Killed                  tail -f test.log[[email protected] ~]# lsof | grep test.log[[email protected] ~]# 

[[email protected] ~]# df -hFilesystem            Size  Used Avail Use% Mounted on/dev/mapper/VolGroup00-LogVol00                       28G   14G   12G  54% //dev/sda1              99M   13M   82M  14% /boottmpfs                1006M     0 1006M   0% /dev/shm[[email protected] ~]# du -sh888K 

At this time, no process occupies the test. log file, and then deletes it from the disk. The super block information of the partition has been changed, and DF is displayed as normal.

4. Notes for routine use

(1) When there is a big gap between du and DF, consider whether the deletion of files is incomplete by running the lsof command and then stop the related process.(That is, if deleted is specified at the end of the lsof file)

(2) You can use the clear file method instead of deleting the file by echo> test. log.

(3) log files with frequent deletion problems are operated in the order of renaming, clearing, and deleting

(4) In addition to RM, some commands will indirectly delete the file. For example, after the gzip command is completed, the original file will be deleted. To avoid deletion problems, make sure that no process opens the file before compression.

(5) how to view files that have been deleted by RM but are still occupied by processes: lsof | grep deleted

 

 

 

 

Reference: http://blog.csdn.net/smstong/article/details/8715650

Note: although the original link content is basically copied, there are some additional gains and experiences in actual testing.

Difference between du and DF and solutions for large gaps