Maximum number of processes and maximum files in Linux

Objective

Linux systems can set restrictions on the use of resources, such as: number of processes, number of file handles, number of connections, and so on.
You should meet in your daily work:

-bash:fork:retry:resource temporarily unavailable

Or

Too many open files

These similar operational errors are due to the current user's number of processes exceeding the limit, which exceeds the limit due to the number of open files for the current user.

The following is a simple introduction to configuring the number of user processes and the number of open files.

A. Ulimit command

Provides control over the resources available to the shell and to processes started by it, in systems that allow such cont Rol.

The Linux man document describes Ulimit in this way: it provides the current shell's ability to control system resources. From here we can analyze the characteristics of Ulimit:

• Controllable system resources: CPU, memory, process, file, etc.
• Scope of control in the current shell session

Syntax for Ulimit:

ulimit [-HSTABCDEFILMNPQRSTUVX [limit]]

Divided into parameters and values two, parameters are used to represent a specific resource type, and the value represents a metric that is restricted to a particular resource.

Command Parameters	Description	Example
-H	Set the hard resource limit once the settings cannot be increased	ULIMIT–HS 64; Limit hard resources, thread stack size is 64K
-S	Setting soft resource limits can be increased after setup, but cannot exceed hard resource settings	ULIMIT–SN 32; Restrict soft resources, 32 file descriptors
-A	Show all current limit information	Ulimit–a; Show all current limit information
-C	The size of the largest core file, in blocks	Ulimit–c Unlimited; No restrictions on the size of the resulting core file
-D	The size of the largest data segment of the process, in Kbytes	Ulimit-d Unlimited; No restrictions on the data segment size of the process
-F	The process can create a maximum value for the file, in blocks	ulimit–f 2048; Limit the maximum file size that a process can create to 2048 blocks
-L	Maximum lockable memory size, in Kbytes	Ulimit–l 32; Limit maximum lockable memory size is Kbytes
-M	Maximum memory size, in Kbytes	Ulimit–m Unlimited; No limit on maximum memory
-N	The maximum number of file descriptors that can be opened	Ulimit–n 128; Limit the maximum number of 128 file descriptors that can be used
-P	The size of the pipe buffer, in Kbytes units	Ulimit–p 512; Limit the size of the pipe buffer to Kbytes
-S	Thread stack size, in Kbytes	Ulimit–s 512; Limit Line stacks size is Kbytes
-T	Maximum CPU elapsed time in seconds	Ulimit–t Unlimited; No limit on maximum CPU occupancy time
-U	Maximum number of processes available to the user Ulimit–u 64	Limit the maximum number of 64 processes users can use
-V	The maximum available virtual memory for the process, in Kbytes	Ulimit–v 200000; Limit the maximum available virtual memory to 200000 Kbytes

Based on the table above, if you need to set the maximum number of processes and the number of open files for the current user, just set the following:

• Ulimit-u 8192 Sets the maximum number of processes for the current user to 8192 and only takes effect in the current shell
• Ulimit-n 8192 Sets the maximum number of open files for the current user 8192, only valid in the current shell

You can also view all restrictions for the current user by using the parameter-a:

[[email protected] ~]$ ulimit -acore file size          (blocks, -c) unlimiteddata seg size           (kbytes, -d) unlimitedscheduling priority             (-e) 0file size               (blocks, -f) unlimitedpending signals                 (-i) 127510max locked memory       (kbytes, -l) 64max memory size         (kbytes, -m) unlimitedopen files                      (-n) 32768pipe size            (512 bytes, -p) 8POSIX message queues     (bytes, -q) 819200real-time priority              (-r) 0stack size              (kbytes, -s) 8192cpu time               (seconds, -t) unlimitedmax user processes              (-u) 4096virtual memory          (kbytes, -v) unlimitedfile locks                      (-x) unlimited

With max user processes and open files, you can see that the maximum number of processes and the maximum open files for the current post user is 32768 and 8192, respectively.

Two. Setting up the configuration file

The Ulimit method is described earlier to modify the limitations of system resources, but there are limitations: Ulimit only takes effect for the current shell session. If you reopen a shell, the configuration again becomes the previous limit. You need to modify the file so that it is configured to persist if it needs to be permanently active.

Configuration files are mainly divided into:

• Writes commands to profile and BASHRC, which is equivalent to automatically changing the limit dynamically when logging in
  
• Add record in/etc/security/limits.conf (requires reboot to take effect, and seesion in/etc/pam.d/is used to limit module)

The first approach is well understood, that is, each time the shell user logs in, the current user's profile execution ulimit command is loaded to modify the resource limit.
Below is the main introduction: How to Modify/etc/security/limits.conf.

#Each Line describes a limit for a user in the form:##<domain> <type> <item> <value># #Whe Re:#<domain> can be:#-a user name#-a group name, with @group syntax#-the wildcard *, for Default entry#-The wildcard%, can be also used with%group syntax,# for Maxlogin limit##<type> C An has the values:#-"soft" for enforcing the soft limits#-' hard ' for enforcing hard limits##<ite        M> can be one of the following:#-core-limits the core file size (KB) #-Data-max data Size (KB) # -Fsize-maximum FileSize (KB) #-Memlock-max locked-in-memory address Space (KB) #-Nofile-max nu Mber of open File descriptors#-Rss-max resident set Size (KB) #-Stack-max stack size (KB) #-C Pu-max CPU Time (MIN) #-Nproc-max number of processes#-as-address space Limit (KB) #-Maxlog Ins-max Number of LogiNS for this user#-Maxsyslogins-max number of logins in the system#-priority-the priority to run user Process with#-Locks-max number of file locks the user can hold#-sigpending-max number of pending SI gnals#-Msgqueue-max memory used by POSIX message queues (bytes) #-Nice-max Nice priority allowed to r         Aise to values: [ -20, 19]#-Rtprio-max realtime priority##<domain> <type> <item> <value>##* Soft core 0#* hard rss 10000# @student ha             Rd Nproc 20# @faculty soft nproc 20# @faculty hard Nproc 50#ftp Hard Nproc 0# @student-maxlogins 4# End of file* soft nofile 32768* hard Nofile 6553 6#use for Oraclepostgres soft nproc 2047postgres hard nproc 16384postgres soft nofile 2048postgres hard Nofile 65536postgr Es Soft Stack 10240

The configuration meaning of the file can be seen from the note description of the file, and the configuration is divided into four parts:

• Domain section: Used to represent a user name or group name. @ followed by a user name or group name indicating the user name or group name; * indicates all users
• Type section: Represents the resource level, with two values soft and hard. Soft says that at the software level, hard
• Item section: Represents a resource type that can be restricted. Core limits the size of the file, locks indicates the maximum number of file locks a user can hold, Nproc represents the maximum number of processes, and Nofile indicates the maximum number of open file handles
• Value section: Values that limit the number of resources

As the above configuration: # @student Hard Nproc 20, indicates that student users on the hardware of the largest number of processes only 20,* soft Nofile 32768 indicates that the maximum number of file open for all users on the software is 32768.

Only modify this file can not complete the configuration, but also need to modify the/etc/security/limits.d/90-nproc.conf configuration file, you may first view the contents of the following file:

# Default limit for number of user‘s processes to prevent# accidental fork bombs.# See rhbz #432903 for reasoning.*          soft    nproc     8192root       soft    nproc     unlimited

As you can see from the comments, it is the default limit for user processes. From the configuration, you can see that the root user does not limit the number of processes, and the maximum number of processes for other users is 8192.

About why Linux has both/etc/security/limit.conf and a limits.d/90-nproc.conf, the problem cannot be answered because of the limited capacity of the individual. Readers are interested in self-study, generous enlighten.

Reference

The nproc problem of ulimit limitation

Maximum number of processes and maximum files in Linux