Understanding and modification of the maximum number of open files for applications under Linux

A Java program running on a Linux system has been running for some time and there is an exception to "Too many open files".

This situation is common in high concurrency access to the file system, multi-threaded network connections and other scenarios. Programs frequently accessed files, sockets in Linux are file files, the system needs to record each current Access file name, location, access authority and other related information, such an entity is called file entry. The "Open Files table" (the orange logo in the figure) stores these file entry and is managed linearly in the form of an array. The filename descriptor (file descriptor) is the pointer to the open files table, which is the subscript index of the open files table, associating each process with the file it accesses.

Each process has a file descriptor table that manages all files accessed by the current process (open or create) with file descriptors associated with the file entry of the files in the Open Files table. Details do not table, for open files table can accommodate how many file entry. The Linux system configures file limits for the Open files table, and if the configuration value is exceeded, requests for additional file actions are rejected and too many open files exception is thrown. This restriction has a system-level and user-level divide.

System level:

System-level settings are valid for all users. There are two ways to view the maximum file limit of a system

1 Cat/proc/sys/fs/file-max

2 sysctl-a View Results Fs.file-max the number of configurations in this item

If you need to increase the number of configurations, modify the/etc/sysctl.conf file, configure the Fs.file-max property, and add if the property does not exist.

Use Sysctl-p when configuration is complete to notify the system to enable this configuration

User level:

Linux limits the number of connected files per logged-on user. You can view the currently valid settings by Ulimit-n. If you want to modify this value, use the ulimit-n command.

For the increase in the file descriptor, the data recommendation is referred to as a power of 2. If the current file descriptor number is 1024, it can be increased to 2048, if not enough, to 4096, and so on.

In the presence of too many open files problem, first find out the main reason. The biggest possibility is that the open file or socket does not shut down properly. To locate whether the problem is caused by a Java process, view the current process occupancy file descriptor through the Java process number:

Lsof-p $java _pid The specific properties of each file descriptor

Lsof-p $java _pid | Wc-l the total amount of FD in the current Java Process file descriptor table

Analyze the results of the command to determine if the problem is caused by an abnormal release of resources.

If we are just ordinary users, only temporarily modify the Ulimit-n, you can directly modify the shell command (ulimit-n 1024000). But this setting is temporarily reserved! When we exit bash, the value is restored to the original value.

If you want to permanently modify the ulimit, you need to modify the/etc/security/limits.conf.

Vim/etc/security/limits.conf

# Add the following line

* Soft Nofile 2048

* Hard Nofile 2048

Here is the description:

* representative for all users

Noproc represents the maximum number of processes

Nofile is the maximum number of file open

Add Formatting:

[username | @groupname] Type resource limit

[username | @groupname]: Set the user name that needs to be restricted, the group name is preceded by @ and user name is distinguished. You can also use the wildcard character * to make all user restrictions.

Type: There are Soft,hard and-,soft that refer to the setting value that the current system is in effect. The hard indicates the maximum value that can be set in the system. The soft limit cannot be higher than the hard limit. Use-Indicates that both soft and hard values are set.

Resource

Core-Limit kernel file size (KB)

Date-Maximum data size (KB)

Fsize-Maximum file size (KB)

Memlock-Maximum locked memory address space (KB)

Nofile-Maximum number of open files

RSS-Maximum persistent setting size (KB)

Stack-Maximum stack size (KB)

CPU-Maximum CPU time in minutes

Noproc-Maximum number of processes

As-address space restrictions

Maxlogins-Maximum number of logins allowed by this user

Instance:

Username Soft Nofile 2048

Username Hard Nofile 2048

@groupname Soft Nofile 2048

@groupname Hard Nofile 2048