Before netstat showed the bug of the program name, it was found that there was a bug in the netstat display program name, and the cause was not thoroughly checked. (If you do not have the permission to view the data, it is irrelevant to the display.-_-B)
Some of you have taken a deeper look at the following:
Http://www.makeinstall.cn/index.php/archives/156
Netstat and web host socket File Analysis (2)
Reprinted, please specify the source, fromWww.makeinstall.cn
In the previous article, we found a bug in netstat, that is, the author did not fully consider that scoket's inode may be large, and even exceed the maximum value of Int. However, even more interesting problems arise: Why is the socket inode number on the Web Front-end machine as large as ten digits, while almost all other machines are eight digits.
3. Analysis of VFS and socket inode
First, we need to know what the socket inode is.
Inode of the socket file exists in the Linux VFS Virtual File System. VFS is a software Bond Layer on a heterogeneous File System. It enables open (), read (), write () system calls can work without worrying about the underlying storage media and file system types. Through VFS, an abstract universal access interface shields the differences between the underlying file system and physical media. The Code of each type of file system hides the implementation details. For the VFS layer and other parts of the kernel, each type of file system looks the same.
VFS inode is different from inode in the disk file system. For example, in the ext2 file system, the inode is located in the block group on the disk, with each inode 128 bytes. When dividing a sector, the system will first make a bunch of inode for later use. The number of inode is related to the total number of files and directories that can be created in the system. Each file on the disk has only one inode. Even if there is no data in the file, inode exists.
VFS inode only exists in the memory and can be accessed through inode cache. Each file has an inode node in VFS, including common files, directories, and special files, such as socket and pipe. When a file is required, the system will create an inode data structure for it in the memory. The inode structure will form a linked list, we can traverse this linked list to get the file node we need. However, VFS allocates inode numbers for different files in different ways. The details are as follows:
- For common files in disk file systems such as ext2, each file has an inode on the disk. The inode number is also known. When accessing these files, VFS will allocate VFS inode to the file in the memory, and assign the inode number identified in the disk file system to the inode structure. It can be seen that the inode numbers of common files are not too large.
- For special files such as sockets, there is no inode number source like the disk file, in implementation, the kernel maintains an unsigned long static variable to save the maximum value of the currently assigned inode number. The newly assigned inode number is added with 1. The value of this static variable increases without decreasing until the machine restarts.
Knowing the inode allocation method of the socket file, we can assume that the socket file is frequently created (and closed) on a machine ), the inode number corresponding to the new socket file will increase at a fast speed. After a long enough time, the inode number will exceed the value of int_max, if netstat-NLP is run, the program name cannot be obtained.
This assumption does occur on the Web Front-end machine. The Web full-segment machine is responsible for responding to a large number of user connection requests every day. Each connection needs to establish a socket for communication, which leads to the rapid growth of the socket inode number. The number of socket inode records the maximum value of socket inode at the same time every day within five days. The analysis shows that the number of appointments increases by about 70,000,000 every day.
Obviously, the growth of socket inode is consistent with the distribution of front-end machine traffic in various time periods. This also proves that the large traffic on the front-end machine causes extremely frequent creation of socket files, resulting in a large inode on the front-end machine.
Too large socket not only affects the display result of netstat, will it exceed the maximum value of unsigned long? The answer is no. On 64-bit online machines, the maximum value of unsigned long is 20 digits. It is impossible to reach ulong_max at 70,000,000 increments every day.
4. Solution
If netstat-NLP cannot display PID/program name due to too large socket, is there any solution or alternative?
- Modify the netstat source code to remove the restriction on the maximum value of socket inode;
- Use the lsof command to view the process listening to a port, for example, $/usr/sbin/lsof-I: Port Number