Analysis of crontab core in linux

Next, I will take a look at the analysis of the crontab core issue in linux. I hope this article will help you.

After receiving a report from my colleagues, the program crash again, but the core cannot be found, and the core should be generated. The system must have a bug.

First, we inspected the system:

1. find the pid Number of another program, then cat/proc/pid/limits | grep core, and return the Max core file size 0 unlimited bytes. If there is a big problem, the Soft Limit value is actually 0, continue to find the reason.

2. Run ulimit-c with the program user and return unlimited. The system settings are correct.

3. Run sysctl-a | grep core _ to check whether the core. % p. % e is returned by core_pattern. The directory does not exist or has permission issues.

4. ssh XX. XX "ulimit-c", remotely log on to and execute the command, and return unlimited, eliminating the problem of not playing core in the strange thing world I mentioned earlier.

I checked all the basic questions. It seems that there is no problem, but I did not hit the core. What is the problem? Tangled. I checked it again.

The problem still persists, but some servers and programs report that the problem is normal .... (Useless) as mentioned in previous articles, ulimit-c unlimited and then Restart sshd. In this way, this parameter is also available for connections over ssh by default. That is to say, the ulimit value is 0, which is probably inherited from the parent program at the upper level. check that the startup time of the program is at the same time point. Search for the bash record and cron record. No one can manually set the ulimit core value and start the game, only cron records restart the game. it is a crontab problem. ps-ef | grep crond finds the crontab process ID, then enters the proc directory to view its limits file, and finds that the program is indeed 0, that is to say, crond is set to do not hit the core, and then any program he starts will not hit the core unless it is manually stated, and then simple verification:

Add the scheduled task run/lib64/libss to the crontab. so.2: This is a database file. If you run it directly, a core error is reported, but no core is entered in crontab. This is the problem: crontab does not cause a core issue.

But why? Why is the crond program not set as a core program? It is reasonable to say that it inherits the bash unlimited value unless it has been set separately, remember that the crond variable can be set in/etc/crontab, and manually specify the unlimited value in it to be the same and invalid. Check/etc/init. d/crond Startup File. It is found that the setting file/etc/sysconfig/crond will be loaded, and then manually specified. The result is still invalid. in/etc/init. in the d/crond file, set the "breakpoint" in several places and add cat/proc/$/limits | grep core. The unlimited value is returned, which is a bit confusing.

When daemon starts crond, it finds some key points and finds the source of daemon:/etc/rc. d/init. the d/functions file contains the definition of the daemon function, and there is a key command line 239 with corelimit = "ulimit-S-c $ {DAEMON_COREFILE_LIMIT: -0} ", and the DAEMON_COREFILE_LIMIT variable has no place to set, think of the problem of the previously configured snmp log, add DAEMON_COREFILE_LIMIT = unlimited to/etc/sysconfig/crond and restart crond. however, this is a program started for crontab. In addition, we often use services to start some programs, and/etc/rc will be loaded in the service by default. d/init. d/functions. If you want to completely eliminate the possibility that the program started with the system service does not hit the core, directly set corelimit = "ulimit-S-c $ {DAEMON_COREFILE_LIMIT: -0} "this line can be commented out.

Re-set crontab and run the/lib64/libss. so.2 file. We found that the crash hit the core smoothly, and the program started by crontab can hit the core smoothly. It seems that the problem has been solved.

But again, why do some projects on some servers use crontab to restart the program? Tragedy, continue to find problems ...... (Business problem, omitted), problem found, because some programs are another parent program daemon, crontab restarts those games simply kill the program, and then the parent program starts automatically, then, the parent program is started manually and can be used to play the core normally. Therefore, this problem does not exist. it is reasonable to say that this situation has been explained again, but with some exceptions, there is a project not started with a daemon, but without this bug, it cannot be explained, the tragedy, continue to find the problem.

The entire process almost re-emerged, but still did not find an explanation, and a new idea was floated. At first, I doubted whether the developers had manually set the ulimit value in the program, but no. If there is no such thing, you can't get through the explanation. Then, we plan to perform a simple analysis on the binary file:
On the server, run the strings command to check whether there are ulimit and core texts. The result is really found. There are texts such as set corefile size, which is definitely the problem, continue to look for the project developers to understand the situation. They have added the setrlimit function to the program, which is used to set the maximum resource usage limit for the task process. this is the problem. this value can also be set for common users because it is for programs or subprograms started by themselves.

OK. I have explained all three other core issues. Here is a summary:

1. When crontab is started, the system function obtains the value of core 0 from the system function. As a result, all programs started do not play core.

2. The core is enabled for the daemon, and all programs started by the daemon directly inherit this attribute.

3. The program can set another core value.

In addition, to check whether a running program can generate a core, the limits file in the pid folder in the proc file system is used instead of making it crash. this is the focus of the previous article. powered by liu.