No space is released after CentOS deletes the file.

No space is released after CentOS deletes the file.

Today, I found that the home space of a server is full, so I want to clear useless files. After I delete the files, I find that the available space has not changed.
OS: centos6.0

Symptom:

Disk space usage:

[Root @ ticketb ~] # Df-h
Filesystem Size Used Avail Use % Mounted on
/Dev/sda1 981 M 203 M 729 M 22%/
None, 16G, 0, 16G, 0%/dev/shm
/Dev/sda9 2.9G 37 M 2.7G 2%/tmp
/Dev/sda7 4.9G 1.9G 2.7G 42%/usr
/Dev/sda8 2.9G 145 M 2.6G 6%/var
/Dev/mapper/vghome-lvhome
20G, 19G, 11 M, 100%/home
/Dev/mapper/vgoradata-lvoradata
144G 48G 90G 35%/u01/oradata
/Dev/mapper/vgbackup-lvbackup
193G 7.8G 175G 5%/u01/backup

Use the following command to find useless files and then delete them.

[Root @ ticketb ~] # Find/home/oracle/admin/dbticb/udump/-name "dbticb _ *. trc"-mtime + 50 | xargs rm-rf

Then, check the disk space usage and find no/home space has not changed.

[Root @ ticketb ~] # Df-h
Filesystem Size Used Avail Use % Mounted on
/Dev/sda1 981 M 203 M 729 M 22%/
None, 16G, 0, 16G, 0%/dev/shm
/Dev/sda9 2.9G 37 M 2.7G 2%/tmp
/Dev/sda7 4.9G 1.9G 2.7G 42%/usr
/Dev/sda8 2.9G 145 M 2.6G 6%/var
/Dev/mapper/vghome-lvhome
20G, 19G, 11 M, 100%/home
/Dev/mapper/vgoradata-lvoradata
144G 48G 90G 35%/u01/oradata
/Dev/mapper/vgbackup-lvbackup
193G 7.8G 175G 5%/u01/backup

This is depressing. It is clear that the file has been deleted. Why is the space not released? The rm command should be deleted directly. Check what space is occupied under/home.

[Root @ ticketb ~] # Du-h -- max-depth = 1/home
16 K/home/lost + found
2.6 GB/home/oracle
2.6 GB/home

The space has been released,

Reason for not releasing disk space:

In Linux or Unix systems, deleting a file through rm or file manager will remove the link from the directory structure of the file system. However, if the file is
Open (a process is in use), the process will still be able to read the file, and the disk space will be occupied all the time. I deleted the oracle Alert log File.
The deleted file should be in use.

Solution

First, obtain a list of files that have been deleted but are still occupied by the application, as shown below:

[Root @ ticketb ~] # Lsof | grep deleted
Oracle 12639 oracle 5 w REG 253,0 648 215907/home/oracle/admin/dbticb/udump/dbticb_ora_12637.trc (deleted)
Oracle 12639 oracle 6 w REG 253,0 16749822091 215748/home/oracle/admin/dbticb/bdump/alert_dbticb.log (deleted)
Oracle 12639 oracle 7u REG 36282, 0 0/home/oracle/product/10.2.0/db_1/dbs/lkinstdbticb (deleted)
Oracle 12639 oracle 8 w REG 16749822091, 0 215748/home/oracle/admin/dbticb/bdump/alert_dbticb.log (deleted)
Oracle 12641 oracle 5 w REG 253,0 648 215907/home/oracle/admin/dbticb/udump/dbticb_ora_12637.trc (deleted)
Oracle 12641 oracle 6 w REG 253,0 16749822091 215748/home/oracle/admin/dbticb/bdump/alert_dbticb.log (deleted)
.
.
Oracle 23492 oracle 6 w REG 253,0 16749822091 215748/home/oracle/admin/dbticb/bdump/alert_dbticb.log (deleted)
Oracle 23492 oracle 7u REG 36282, 0 0/home/oracle/product/10.2.0/db_1/dbs/lkinstdbticb (deleted)
Oracle 23492 oracle 8 w REG 16749822091, 0 215748/home/oracle/admin/dbticb/bdump/alert_dbticb.log (deleted)
Oracle 23494 oracle 10u REG 36307, 0 0/home/oracle/product/10.2.0/db_1/dbs/lkinstrmandb (deleted)

The output result shows that/home/oracle/admin/dbticb/bdump/alert_dbticb.log is still in use and has not been released.

How can we release a process?

One way is to kill the corresponding process or stop the application that uses the file so that the OS can automatically reclaim disk space.

This file is used by many processes in my environment. It is a little troublesome to stop the process, and there is a great risk.

When a file is opened in linux, the linux kernel creates a pid for each process in the/proc/"/proc/nnnn/fd/directory (nnnn is pid )"
A directory named after the name is used to save information about a process. Its fd subdirectory stores the fd (fd: file descriptor) of all files opened by the process ).

By truncating files in the proc file system, the kill process forcibly requires the system to recycle the files allocated to being used.
This is an advanced technology that is only used when the Administrator determines that it will not affect running processes. The Application
Is not supported. When a file in use is truncated, unpredictable problems may occur.

So I still use stop application to solve this problem.

Restart the oracle database and find that the space corresponding to/home/oracle/admin/dbticb/bdump/alert_dbticb.log is released

Check the usage of the disk space and find that the space has been recycled.

[Root @ ticketb ~] # Df-h
Filesystem Size Used Avail Use % Mounted on
/Dev/sda1 981 M 203 M 729 M 22%/
None, 16G, 0, 16G, 0%/dev/shm
/Dev/sda9 2.9G 37 M 2.7G 2%/tmp
/Dev/sda7 4.9G 1.9G 2.7G 42%/usr
/Dev/sda8 2.9G 145 M 2.6G 6%/var
/Dev/mapper/vghome-lvhome
20G 2.6G 16G 15%/home
/Dev/mapper/vgoradata-lvoradata
144G 48G 90G 35%/u01/oradata
/Dev/mapper/vgbackup-lvbackup
193G 7.8G 175G 5%/u01/backup

OK, solve the problem, and then finish the work.

Bytes -------------------------------------------------------------------------------------------------

Learn the lsof command

Lsof full name list opened files, that is, list files already opened in the system. We all know that in linux, everything is a file,
The device is a file, the directory is a file, and even sockets is a file. Therefore, using the lsof command is very helpful for daily linux management.

Lsof is one of the most common commands in linux. The output format is as follows:

Reference
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME

The following fields are common: more visible manual.

1. COMMAND
The default command name is 9 characters long. You can use the + c parameter to specify the display width. If the parameter followed by + c is zero, the full name of the command is displayed.
2. PID: ID of the process
3. PPID
The IP number of the parent process, which is not displayed by default. This parameter can be enabled when the-R parameter is used.
4. PGID
The ID of the Process Group, which is not displayed by default. It can be opened when the-g parameter is used.
5. USER
Command Execution UID or the name of the user logged on to the system. The user name is displayed by default. When the-l parameter is used, the UID is displayed.
6. FD
Is the File Descriptor number of the File, or the following content:
(It is difficult to translate the corresponding meaning here)

Reference
Cwd current working directory;
Lnn library references (AIX );
Jld jail directory (FreeBSD );
Ltx shared library text (code and data );
Mxx hex memory-mapped type number xx.
M86 DOS Merge mapped file;
Mem memory-mapped file;
Mmap memory-mapped device;
Pd parent directory;
Rtd root directory;
Tr kernel trace file (OpenBSD );
Txt program text (code and data );
V86 VP/ix mapped file;

File Descriptor number Display Mode:

Reference
R for read access;
W for write access;
U for read and write access;
N for a Solaris NFS lock of unknown type;
R for read lock on part of the file;
R for a read lock on the entire file;
W for a write lock on part of the file;
W for a write lock on the entire file;
U for a read and write lock of any length;
U for a lock of unknown type;
X for an SCO OpenServer Xenix lock on part of the file;
X for an SCO OpenServer Xenix lock on the entire file;
Space if there is no lock.

7. TYPE

Reference
IPv4 packet;
IPv6 uses an IPv6 package, even if the address is IPv4, it is displayed as IPv6, and the address mapped to IPv6;
DIR directory
LINK file

For details, see more comments in manual.

8. DEVICE
Device ID indicated by character special and block special
9. SIZE
File size. If it cannot be expressed in size, it will be left blank. Use-s parameter control.
10. NODE
The node Code or protocol of the local file, such as TCP.
11. NAME
The full path of the mount point and file (the link will be resolved to the actual path), or the address, port, and status of both parties.

Common examples:

1. display the process of enabling the file/home/oracle/10.2.0/db_1/bin/tnslsnr

[Root @ svr-db-test ~] # Lsof/home/oracle/10.2.0/db_1/bin/tnslsnr
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Tnslsnr 3520 oracle txt REG 431062, 5 11408866/home/oracle/10.2.0/db_1/bin/tnslsnr

2. Know what programs are running on port 22

[Root @ svr-db-test ~] # Lsof-I: 22
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Sshd 3101 root 3u ipv6-8670 TCP *: ssh (LISTEN)
Sshd 4545 root 3u IPv6 4237972 TCP 203.aibo.com: ssh-> win-avbmq9e8ka7.gdgg.local: nsjtp-ctrl (ESTABLISHED)

3. display the files opened by the init process

[Root @ svr-db-test ~] # Lsof-c init
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Init 1 root cwd DIR 253,0 4096 2/
Init 1 root rtd DIR 253,0 4096 2/
Init 1 root txt REG 43496, 0 524446/sbin/init
Init 1 root mem REG 253,0 130448 917826/lib64/ld-2.5.so
Init 1 root mem REG 253,0 1678480 917827/lib64/libc-2.5.so
Init 1 root mem REG 253,0 23520 917686/lib64/libdl-2.5.so
Init 1 root mem REG 247528 917844/lib64/libsepol. so.1
Init 1 root mem REG 95480, 0 917845/lib64/libselinux. so.1
Init 1 root 10u FIFO 2311/dev/initctl

4. Check the files opened by the process with process number 1.

[Root @ svr-db-test ~] # Lsof-p 1
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Init 1 root cwd DIR 253,0 4096 2/
Init 1 root rtd DIR 253,0 4096 2/
Init 1 root txt REG 43496, 0 524446/sbin/init
Init 1 root mem REG 253,0 130448 917826/lib64/ld-2.5.so
Init 1 root mem REG 253,0 1678480 917827/lib64/libc-2.5.so
Init 1 root mem REG 253,0 23520 917686/lib64/libdl-2.5.so
Init 1 root mem REG 247528 917844/lib64/libsepol. so.1
Init 1 root mem REG 95480, 0 917845/lib64/libselinux. so.1
Init 1 root 10u FIFO 2311/dev/initctl

5. display processes belonging to 3520

[Root @ svr-db-test ~] # Lsof-g 3520
COMMAND PID PGID USER FD TYPE DEVICE SIZE NODE NAME
Tnslsnr 3520 3520 oracle cwd DIR 253,5 4096 11059201/home/oracle
Tnslsnr 3520 3520 oracle rtd DIR 253,0 4096 2/
Tnslsnr 3520 3520 oracle txt REG 253,5 431062 11408866/home/oracle/10.2.0/db_1/bin/tnslsnr
Tnslsnr 3520 3520 oracle mem REG 253,0 130448 917826/lib64/ld-2.5.so
Tnslsnr 3520 3520 oracle mem REG 253,0 1678480 917827/lib64/libc-2.5.so
Tnslsnr 3520 3520 oracle mem REG 253,0 23520 917686/lib64/libdl-2.5.so
Tnslsnr 3520 3520 oracle mem REG 253,0 615136 917834/lib64/libm-2.5.so
Tnslsnr 3520 3520 oracle mem REG 253,0 141208 917829/lib64/libpthread-2.5.so
Tnslsnr 3520 3520 oracle mem REG 253,0 109824 917839/lib64/libnsl-2.5.so
Tnslsnr 3520 3520 oracle mem REG 253,5 20706622 11405436/home/oracle/10.2.0/db_1/lib/libclntsh. so.10.1
Tnslsnr 3520 3520 oracle mem REG 253,5 3803097 11410641/home/oracle/10.2.0/db_1/lib/libnnz10.so
Tnslsnr 3520 3520 oracle mem REG 253,5 83493 11407251/home/oracle/10.2.0/db_1/lib/libons. so
Tnslsnr 3520 3520 oracle mem REG 253,0 53880 917532/lib64/libnss_files-2.5.so
Tnslsnr 3520 3520 oracle mem REG 253,5 8545 11407615/home/oracle/10.2.0/db_1/lib/libskgxn2.so
Tnslsnr 3520 3520 oracle mem REG 253,5 513705 11410332/home/oracle/10.2.0/db_1/lib/libocrutl10.so
Tnslsnr 3520 3520 oracle mem REG 253,5 636161 11410330/home/oracle/10.2.0/db_1/lib/libocr10.so
Tnslsnr 3520 3520 oracle mem REG 253,5 657825 11410331/home/oracle/10.2.0/db_1/lib/libocrb10.so
Tnslsnr 3520 3520 oracle mem REG 253,5 1745769 11410365/home/oracle/10.2.0/db_1/lib/libhasgen10.so
Tnslsnr 3520 3520 oracle mem REG 253,5 61985 11410366/home/oracle/10.2.0/db_1/lib/libclsra10.so
Tnslsnr 3520 3520 oracle 0u CHR 2553/dev/null
Tnslsnr 3520 3520 oracle 1u CHR 2553/dev/null
Tnslsnr 3520 3520 oracle 2u CHR 2553/dev/null
Tnslsnr 3520 3520 oracle 3 w REG 253,5 318853012 11633459/home/oracle/10.2.0/db_1/network/log/listener. log
Tnslsnr 3520 3520 oracle 4r FIFO 15661 pipe
Tnslsnr 3520 3520 oracle 5r REG 253,5 11776 11410579/home/oracle/10.2.0/db_1/network/mesg/nlus. msb
Tnslsnr 3520 3520 oracle 6r REG 253,5 46592 11407160/home/oracle/10.2.0/db_1/network/mesg/tnsus. msb
Tnslsnr 3520 3520 oracle 7 w FIFO 15662 pipe
Tnslsnr 3520 3520 oracle 8u IPv4 15665 TCP 203.aibo.com: ncube-lm (LISTEN)
Tnslsnr 3520 3520 oracle 9u unix 0xffff81021b7d6980 15666/var/tmp/. oracle/s #3520.1
Tnslsnr 3520 3520 oracle 10u unix 0xffff81021b7d66c0 15668/var/tmp/. oracle/s #3520.2

6. Search by Folder/home/oracle, but do not open sub-directories to display files opened by processes in the directory

[Root @ svr-db-test ~] # Lsof + d/home/oracle
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Tnslsnr 3520 oracle cwd DIR 253,5 4096 11059201/home/oracle

7. Open the/home/oracle folder and Its subdirectory search to display files opened by processes in the directory.

[Root @ svr-db-test ~] # Lsof + D/home/oracle

Too much content is displayed.

8. lsof-I is used to display qualified Processes

Syntax: lsof-I [46] [protocol] [@ hostname | hostaddr] [: service | port]

46 --> IPv4 or IPv6

Protocol --> TCP or UDP

Hostname --> Internet host name

Hostaddr --> IPv4 location

Service --> service name in/etc/service (more than one)

Port --> port number (either)

Example:

[Root @ svr-db-test ~] # Lsof-itcp@192.168.2.245: 1521-n
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Oracle 15633 oracle 16u IPv4 4069605 TCP 192.168.2.203: 31580-> 192.168.2.245: ncube-lm (ESTABLISHED)

Or

[Root @ svr-db-test ~] # Lsof-itcp@192.168.2.245: 1521
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Oracle 15633 oracle 16u IPv4 4069605 TCP 203.aibo.com: 31580-> 192.168.2.245: ncube-lm (ESTABLISHED)

Lsof-n does not convert the IP address to the hostname. The-n parameter is not added by default.

9. Display opened files of a user (or files opened by the user's execution Program)

[Root @ svr-db-test ~] # Lsof-u oracle
Or
[Root @ svr-db-test ~] # Lsof-u 0

10. Print only processes to facilitate shell script calls.

[Root @ svr-db-test ~] # Lsof-tc sshd
3101
4545

Follow:

Process Debugging commands: truss, strace, and ltrace

The process cannot be started, the software runs slowly, and the program's "SegmentFault" is a headache for every Unix system user, these problems can be quickly diagnosed by using truss, strace and ltrace, which are commonly used debugging tools ".