Set maximum file open number under Linux nofile and Nr_open, File-max instructions

 

In the development of operations we often encounter similar to "Socket/file:can" T open so many files, "Unable to open more processes", or coredump too large problems, which can be set resource constraints to solve. Today, when I was teaching a customer how to set the maximum number of files, the search method on the Internet found that the various statements were inconsistent and wrote this document.

Usually set up system resources for a Linux user, we already know can use the Ulimit command to view and set.

Table 1. Ulimit parameter Description

options [Options]	Meaning	Example
-H	Set the hard resource limit once the setting cannot be increased.	ULIMIT–HS 64; Limit hard resources, thread stack size is 64K.
-S	Setting a soft resource limit can be increased after setup, but cannot exceed hard resource settings.	ULIMIT–SN 32; Restrict soft resources, 32 file descriptors.
-A	Displays all current limit information.	Ulimit–a; Displays all current limit information.
-C	The size of the largest core file, in blocks.	Ulimit–c Unlimited; the size of the resulting core file is not limited.
-D	The size of the largest data segment of the process, in Kbytes.	Ulimit-d Unlimited; The data segment size of the process is not limited.
-F	The process can create a maximum value for the file, in blocks.	ulimit–f 2048; Limit the maximum file size that a process can create to 2048 blocks.
-L	Maximum lockable memory size, in Kbytes.	Ulimit–l 32; Limit maximum lockable memory size to Kbytes.
-M	The maximum memory size, in Kbytes.	Ulimit–m Unlimited; No limit on maximum memory.
-N	The maximum number of file descriptors can be opened.	Ulimit–n 128; Limit the maximum number of 128 file descriptors that can be used.
-P	The size of the pipe buffer, in Kbytes.	Ulimit–p 512; Limit the size of the pipe buffer to Kbytes.
-S	The thread stack size, in Kbytes.	Ulimit–s 512; The size of the limit line stacks is Kbytes.
-T	Maximum CPU elapsed time, in seconds.	Ulimit–t Unlimited; There is no limit to the maximum CPU occupancy time.
-U	The maximum number of processes available to the user.	Ulimit–u 64; Limit the maximum number of users to use 64 processes.
-V	The maximum available virtual memory for the process, in Kbytes.	Ulimit–v 200000; Limit the maximum available virtual memory to 200000 Kbytes.

Of course, we all know that most of Linux command settings are temporary, and the Ulimit command is only valid for the current terminal, if it is necessary to be permanent, we have two methods, one is to write the command into profile and BASHRC, In some pages it is not right to write to rc.local, but also to add a record in limits.conf (requires a reboot to take effect, and seesion in/etc/pam.d/to use the Limit module). The next discussion is about the limit of the maximum number of open files for limits.conf users.

For the user to open the maximum number of files limit, in the limits.conf corresponding to the nofile, whether it is the man manual or file description is just a word "maximum number of open files", it actually corresponds to a single process can open the maximum file count, Usually for the sake of convenience, we would like to remove its restrictions, according to the Man Handbook, "values-1, unlimited or infinity indicating no Limit",-1, unlimited, infinity are not limited, However, when you actually set this value to Nofile, you will find that you cannot log on to the system when you restart.

This shows that Nofile has an upper limit, while testing with Ulimit:

#ulimit-N Unlimited

Bash:ulimit:open Files:cannot Modify limit: Operations not allowed

Write a simple for loop to draw:

#for V in ' seq 100000 10000000 ';d o ulimit-n $V; [[$?! = 0]]&&break;done

Then execute ulimit-n, you can see that 1048576 is the maximum value of nofile, but why is this value? 1048576 is 1024*1024, of course, this does not have any egg to use ... And then we'll see that this value is actually defined by the kernel parameter nr_open:

# Cat/proc/sys/fs/nr_open

1048576

To this we will talk about Nr_open, and File-max, on the internet when it comes to setting the maximum number of files occasionally some posts also said to modify the File-max, literally see File-max really like the maximum number of files, and in the Linux kernel document they are two explanations are:

File-max:

The value in File-max denotes the maximum number of file-
Handles that the Linux kernel would allocate. When you get lots
of error messages about running out of file handles, you might
Want to increase this limit

Execution: Grep-r Memtotal/proc/meminfo | awk ' {printf ("%d", $2/10)} ', can be seen similar to File-max;

Nr_open:

This denotes the maximum number of file-handles a process can
Allocate. Default value is 1024*1024 (1048576) which should be
enough for most machines. Actual limit depends on Rlimit_nofile
Resource limit.

Well, to this I am a bit dizzy, file-handles (that is, file handles), and then compared to the unix/linux we touch more is the file Discriptor (FD, that is, file descriptor), through the Baidu search, It seems like File-handle is a file discrptor in Windows, but we're talking about Linux, and then Google, we can pinpoint the difference between these two concepts in C. According to their discussion, File-handle should be a high-level object, called using functions such as Fopen,fread, and FD is an underlying object that can be invoked through functions such as open,read.

In this case, we should be able to make a general conclusion, File-max is the maximum number of files that can be allocated by the kernel, and Nr_open is the maximum number of files that a single process can allocate. So when we use Ulimit or limits.conf to set, if we want to exceed the default value of 1048576, we need to increase the Nr_open value first (sysctl-w fs.nr_open=100000000 or write directly to the sysctl.conf file). Of course millions other single process maximum File-handle open number should also enough?? .

Because the author level is limited, the content inevitably has the omission, also has the File-max default value to be smaller than the Nr_open value to be small also did not further investigate, welcome everybody to discuss the exchange.

Set maximum file open number under Linux nofile and Nr_open, File-max instructions