Original posted Address:
https://segmentfault.com/a/1190000006880817
A few days ago to view the error log Heka log, found that the fault information too many open files, it is obvious that the number of open file too much.
The first question is how to view the number of files opened by the current process and the maximum number of open files.
Number of files open for current process
Ls/proc/[pid]/fd|wc-l
The maximum number of open files for the current process
Cat/proc/[pid]/limits|grep Open
You can see the output shown below:
Max open Files 1024 4096 files
Current system maximum number of open files
Ulimit-n
The second question is how do I modify the maximum number of open files on the process.
Find the maximum number of File open settings, this problem is resolved, there are usually the following ways to modify:
1) ulimit-n 102400 is modified directly using the Ulimit command, but this only takes effect on the current session, and the settings are lost after the terminal is closed.
2/etc/security/limitd.conf file to add limits configuration, generally as follows:
* Soft nofile 102400
The specific meaning of the configuration, we search by ourselves. /etc/security/limitd.conf is loaded every time the session is created, so modifying here is a way to make the configuration work for a long time.
3 Modify the startup entry of the shell and put Ulimit-n 102400 in, which is also loaded each time the session is created. Typically/etc/profile files, or/etc/profile.d/limits.sh.
So far, configured, you see through the Ulimit-n the system's maximum number of open files has been in effect. However, the maximum number of file turns to view the process does not change at this point because the value is set when the process is started and must be restarted to take effect.
OK, then restart it, reboot completed, the results are still unchanged. This odd strange, then after a long time to check, the final confirmation of the problem is that the program is managed through Supervisord, that is, this process is the Supervisord process, and supervisord the largest number of files open or old configuration, this time must be restarted Supervisord only can. Later on the Saltstack also encountered the same problem, must be all salt-minion reboot.
When you encounter limits changes do not take effect, please check whether the process is a subprocess, if it is, then the parent process will also be restarted.