Generate core files in Linux

1. Introduction to core files

When a program crashes, it generally generates a core file under the specified directory. The core file is only a memory image (with debugging information added) and is mainly used for debugging.

 

2. enable or disable the generation of core files

Run the following command to prevent the system from generating the core file: ulimit-C 0. Run the following command to check whether the option for generating the core file is Enabled: ulimit-a this command will display all user customization, option-A indicates "all ". You can also modify the system file to adjust the core option. in/etc/profile, this statement is usually used to prohibit the generation of core files. This setting is generally reasonable: # No core files bydefaultulimit-s-c 0>/dev/null 2> & 1 but sometimes for debugging problems during development, you still need to enable the settings generated by the core file in a specific user environment ~ /. Add ulimit to the bash_profile.
-C unlimited allows specific users to generate core files. If ulimit-C 0 is used, it is also prohibited to generate core files, ulimit-C 1024 limits the size of the generated core file to kb.

 

3. Set the core dump file directory and naming rules for core dump.

/Proc/sys/kernel/core_uses_pid can control whether PID is added to the file name of the generated core file as an extension. If it is added, the file content is 1; otherwise, it is 0. /Proc/sys/kernel/core_pattern can be used to set the location or file name of the formatted core file. For example, if the original file content is core-% E, you can modify it as follows: echo "/corefile/Core-% E-% P-% t"> core_pattern controls the generated core files to be stored in the/corefile directory, the generated file name is core-command name-PID-timestamp. The following is the parameter list: % P
-Insert PID into filename add PID % u-insert current uid into filename add current uid % G-insert current GID into filename add current GID % s-insert signal thatcaused the coredump into the filename add resulting in core signal % T-insertunix time that the coredump occurred into filename
Add the Unix time % H-inserthostname where the coredump happened into filename when the core file is generated to add the host name % E-insert coredumpingexecutable name into filename add command name

 

4. Use a Core File

In the directory where the core file is located, Type GDB-C Core. It starts the GNU Debugger to debug the core file and displays the program name that generates the core file, abort the program signal. If you already know which program generates the core file, for example, if myserver crashes to generate core.12345, use this command for debugging: GDB-C Core myserver: how to learn how to use GDB?

 

5. A small method to test and generate core files

Enter the command: Kill-ssigsegv $

Coredump introduction and coredump reason Summary

I wrote a simple introduction to the linuxcore file. Many people will ask what coredump is? What can we do with coredump? Prepare to write a series of such articles to give a detailed introduction to coredump

What is coredump?

Generally, coredmp includes the memory, register status, stack pointer, and memory management information when the program is running. It can be understood that the current state of the program work is stored as a file. A core file is automatically generated when errors occur in many programs and operating systems.

 

How to Use coredump?

Coredump can be used in many occasions. Users who use Linux or Solaris may have this experience. If the system is running some stress tests or the system has a large load, the system will hang or simply system panic. the only thing that can help you analyze and solve problems is coredump.

Currently, many coredump errors may occur in applications.

 

Tool for coredump Analysis

Currently, most UNIX-like operating systems provide tools for analyzing core files, such as GNU binutils binary filedescriptor Library (BFD), GNU Debugger (GDB), and MDB.

Coredump File Format

In Unix-like operating systems, coredump files are saved in EFI format.

Under solairs

Bash-3.2 # file * unix.3 elf 32-bit lsbexecutable 80386 version 1, statically linked, not stripped, no debugginginformation availableunix.4 elf 32-bit LSB executable 80386 version 1, statically linked, not stripped, no debugging information available

 

There are many reasons for coredump. Here we will summarize the reasons based on previous experience:

1 memory access out of bounds

A) array access is out of bounds due to incorrect subscript.

B) when searching for a string, the string Terminator is used to determine whether the string ends, but the string does not use the terminator normally.

C) use strcpy, strcat, sprintf, strcmp, strcasecmp, and other string operation functions to read/write the target string. Functions such as strncpy, strlcpy, strncat, strlcat, snprintf, strncmp, and strncasecmp should be used to prevent read/write from being out of bounds.

 

2 multi-threaded programs use functions that are not thread-safe.

The following reusable functions should be used. Pay special attention to the functions marked in red. They are easy to be used incorrectly:

Asctime_r (3C) gethostbyname_r (3N) getservbyname_r (3N) ctermid_r (3 S) gethostent_r (3N) getservbyport_r (3N) ctime_r (3C) getlogin_r (3C) getservent_r (3n) fgetgrent_r (3C) getnetbyaddr_r (3N) getspent_r (3C) fgetpwent_r (3C) getnetbyname_r (3N) getspnam_r (3C) fgetspent_r (3C)
Getnetent_r (3N) gmtime_r (3C) gamma_r (3 m) getnetgrent_r (3N) lgamma_r (3 m) random (3) getprotobyname_r (3N) localtime_r (3C) random (3) etprotobynumber_r (3N) nis_sperror_r (3N) getauevent_r (3) getprotoent_r (3N) rand_r (3C) getauevnam_r (3) getpwent_r (3C)
Readdir_r (3C) getauevnum_r (3) getpwnam_r (3C) strtok_r (3C) getgrent_r (3C) getpwuid_r (3C) tmpnam_r (3 S) getgrgid_r) ttyname_r (3C) getgrnam_r (3C) getrpcbynumber_r (3N) gethostbyaddr_r (3N) getrpcent_r (3N)

 

3 multi-threaded read/write data is not locked.

For global data that will be accessed by multiple threads at the same time, pay attention to lock protection, otherwise it will easily cause core dump

 

4 Invalid Pointer

A) Use a null pointer.

B) Use Pointer conversion at will. A pointer pointing to a memory segment, unless it is determined that the memory is originally allocated to a structure or type, or an array of this structure or type, otherwise, instead of converting it to a pointer of this structure or type, you should copy this memory to a structure or type and then access this structure or type. This is because if the starting address of this memory segment is not aligned according to this structure or type, it is easy to access it because of Bus Error and core dump.

 

5 Stack Overflow

Do not use large local variables (because all local variables are allocated on the stack). This can easily cause stack overflow, damage the stack and heap structure of the system, and cause inexplicable errors.

 

Linux core dump configuration and debugging

1. Core File generation switch and size limit

---------------------------------

1) Use the ulimit-C command to view the core file generation switch. If the result is 0, this function is disabled and core files are not generated.

2) use the ulimit-cfilesize command to limit the size of the core file (The unit of filesize is Kbyte ). If ulimit-cunlimited is used, the size of the core file is not limited. If the generated information exceeds this size, it will be cropped to generate an incomplete core file. When debugging this core file, GDB will prompt an error.

 

2. Name and path of the Core File

----------------------------

If the core file generated by the system does not contain any other extension names, it is all named core. The new core file will overwrite the original core file.

1)/proc/sys/kernel/core_uses_pid can control whether PID is added to the file name of the core file as an extension. If the file content is 1, the PID is added as the extension, and the generated core file format is core. xxxx. If it is 0, the generated core file is named core.

Run the following command to modify the file:

Echo "1">/proc/sys/kernel/core_uses_pid

 

2) proc/sys/kernel/core_pattern can control the core file storage location and file name format.

Run the following command to modify the file:

Echo "/corefile/Core-% E-% P-% t"> core_pattern, which can generate core files in the/corefile directory, the generated file name is core-command name-PID-timestamp.

The following is a list of parameters:

% P-insert PID into filename add PID

% U-insert current uid into filename add current uid

% G-insert current GID into filename add current GID

% S-insert signal that caused the coredump into the filename added to generate core Signal

% T-insert UNIX time that the coredump occurred into filename UNIX time when the core file is generated

% H-insert hostname where the coredump happened into filename Add the Host Name

% E-insert coredumping executable name into filename add command name

 

3. Use GDB to view the core file:

Here we can dump the core when an error occurs due to the message number.

After a core dump occurs, use GDB to view the content of the core file to locate the line that causes the core dump in the file.

GDB [exec file] [core file]

 

For example:

GDB./test. Core

After entering GDB, run the BT command to check where the program is running and locate the core dump file-> line.

 

4. Use the core file for debugging ON THE DEVELOPMENT BOARD

-----------------------------

If the operating system of the Development Board is also Linux, the core debugging method is still applicable. If GDB is not supported on the Development Board, you can copy the Development Board environment (header files, libraries), executable files, and core files to Linux on the PC and run relevant commands.

Note: The executable file to be debugged must be added to-g during compilation so that the core file can display error information normally!

Notes:

In Linux, pay attention to the following issues to ensure coredump is generated when the program crashes:

1. Ensure that the directory where coredump is stored exists and the process has write permission on the directory. The directory that stores coredump is the current directory of the process, which is generally the directory where the command was issued to start the process. However, if the script is started, the script may modify the current directory, and the current directory of the process will be different from the directory where the script was originally executed. In this case, you can view the target of the "/proc/<process pid>/CWD" symbolic link to determine the real current directory address of the process. You can also view the processes started by the system service in this way.

 

2. If the program calls seteuid ()/setegid () to change the valid user or group of the process, the system will not generate coredump for these processes by default. Many service programs call seteuid (), such as MySQL. No matter which user you use to run mysqld_safe to start MySQL, the effective user of mysqld is always the msyql user. If you run a program with user A, but the user who sees the program in PS is B, then these processes call seteuid. To allow these processes to generate core dump, you need to change the content of the/proc/sys/fs/suid_dumpable file to 1 (usually 0 by default ).

 

3. This is generally known as setting a large enough core file size limit. The core file size generated when the program crashes is the memory size occupied by the program running. However, when a program crashes, the behavior cannot be estimated as usual. For example, errors such as buffer overflow may cause the stack to be destroyed. Therefore, the value of a variable is often changed to a mess, then the program uses this size to apply for memory, which may cause the program to occupy more memory than usual. Therefore, no matter how little memory is occupied during normal running of the program, make sure to generate the core file or set the size limit to unlimited.

 

When our program crashes, the kernel may map the current memory of the program to the core file, so that programmers can find out where the program is faulty. Most often, the errors that almost all c programmers encounter are "segment errors. It is also the most difficult error to identify the cause of the problem. Next we will analyze the generation of core files for "segment errors" and how we can use core files to find the crash location.

 

What is a core file?

When a program crashes, the stored images of the process are copied to the core file in the current working directory of the process. The core file is only a memory image (with debugging information added) and is mainly used for debugging.

When the program receives the following UNIX signal, it will generate a core file:

 

Name

Description

Ansi c posix.1

Svr4 4.3 + BSD

Default action

SIGABRT

Abort)

Terminate w/Core

Sigbus

Hardware faults

Terminate w/Core

Sigemt

Hardware faults

Terminate w/Core

Sigfpe

Arithmetic exception

Terminate w/Core

Sigill

Invalid hardware instruction

Terminate w/Core

Sigiot

Hardware faults

Terminate w/Core

Sigquit

Terminal exit character

Terminate w/Core

SIGSEGV

Invalid Storage Access

Terminate w/Core

Sigsys

Invalid system call

Terminate w/Core

Sigtrap

Hardware faults

Terminate w/Core

Sigxcpu

CPU limit exceeded (setrlimit)

Terminate w/Core

Sigxfsz

Exceeds the file length limit (setrlimit)

Terminate w/Core

In the system default action column, "Terminate w/core" indicates that the stored image of the process is copied to the core file in the current working directory of the process (the file name is core, from this we can see that this feature was a part of UNIX long ago ). Most Unix debugging programs use core files to check the state of a process upon termination.

The generation of core files is not part of posix.1, but the implementation features of many UNIX versions. UNIX 6th does not have check conditions (a) and (B), and its source code contains the following instructions: "If you are looking for protection signals, when you run the-user-id command, a large amount of such signals may be generated ". 4.3 + BSD generates a file named core. prog, where prog is the first 6 Characters of the program name to be executed. It identifies a core file, so it is an improved feature.

The "hardware faults" in the table correspond to the hardware faults defined by the implementation. Many of these names are taken from UNIX earlier implementations on the DP-11. Please refer to the system manual you are using to determine exactly which error types these signals correspond.

These signals are described in detail below.

• SIGABRT generates this signal when calling the abort function. Abnormal Process Termination.

• Sigbus indicates an implementation-defined hardware fault.

• Sigemt indicates a hardware failure that is defined for the implementation.

The emulator trap command from the PDP-11.

• Sigfpe this signal indicates an arithmetic operation exception, such as dividing by 0 and floating point overflow.

• Sigill this signal indicates that the process has executed an illegal hardware command.

 

4.3bsd this signal is generated by the abort function. SIGABRT is currently used here.

• Sigiot indicates a hardware failure that is defined by the implementation.

The Iot name comes from the PDP-11's abbreviation for the input/output trap (input/outputtrap) command. This signal is generated by the abort function in earlier versions of System V. SIGABRT is currently used here.

• Sigquit: this signal is generated when you press the return key (CTRL-/) on the terminal and sent to the frontend.

All processes in the process group. This signal not only terminates the foreground process group (as SIGINT does), but also generates a core file.

• SIGSEGV indicates that the process has an invalid storage access.

The segv name indicates "segment violation (segmentation violation )".

• Sigsys indicates an invalid system call. For some unknown reason, the process executes a System Call Command,

However, the parameter indicating the system call type is invalid.

• Sigtrap indicates an implementation-defined hardware fault.

This signal name comes from the trap command for the PDP-11.

• Sigxcpu svr4 and 4.3 + BSD support the concept of resource restrictions. This signal is generated if the process exceeds its soft c p u time limit.

• If the process of sigxfsz exceeds its soft file length limit, svr4 and 4.3 + BSD generate this signal.

 

Use core files to debug programs

See the following example:

/* Core_dump_test.c */

# Include <stdio. h>

Const char * STR = "test ";

Void core_test (){

STR [1] = 'T ';

}

 

Int main (){

Core_test ();

Return 0;

}

 

Compile:

Gcc-G core_dump_test.c-O core_dump_test

 

If you need to debug the program, add the-G option during GCC compilation, which makes it easier to locate the error when debugging the core file.

Run:

./Core_dump_test

Segment Error

When you run the core_dump_test program, a "segment error" occurs, but no core file is generated. This is because the default core file size is 0, so it is not created. You can use the ulimit command to view and modify the size of the core file.

Ulimit-C 0

Ulimit-C 1000

Ulimit-C 1000

-C specifies the size of the core file, and 1000 specifies the size of the core file. You can also limit the size of the core file, for example:

Ulimit-C Unlimited

Ulimit-C Unlimited

If you want the change to take effect permanently, You need to modify the configuration file, such as. bash_profile,/etc/profile, or/etc/security/limits. conf.

Execute again:

./Core_dump_test

Segment error (coredumped)

Ls core .*

Core.6133

You can see that a core.68.0 file has been created. 6133 is the ID of the process where core_dump_test runs.

Tune the Core File

The core file is a binary file, and a corresponding tool is required to analyze the memory image when the program crashes.

File core.68.0

Core.6133: Elf 32-bit LSB Core File intel80386, Version 1 (sysv), SVR4-style, from 'core _ dump_test'

In Linux, you can use GDB to debug core files.

GDB core_dump_test core.6133

Gnu gdb Red Hat Linux (5.3post-0.20021129.18rh)

Copyright 2003 Free Software Foundation, Inc.

GDB is free software, covered by the gnugeneral Public License, and you are

Welcome to change it and/or distributecopies of it under certain conditions.

Type "show copying" to see theconditions.

There is absolutely no warranty forgdb. Type "show warranty" Fordetails.

This GDB was configured as "i386-redhat-linux-gnu "...

Core was generated by './core_dump_test '.

Program terminated with signal 11, segmentation fault.

Reading symbols from/lib/tls/libc. so.6...... done.

Loaded symbols for/lib/tls/libc. so.6

Reading symbols from/lib/ld-linux.so.2... done.

Loaded symbols for/lib/ld-linux.so.2

#0 0x080482fd in core_test () at core_dump_test.c: 7

7 STR [1] = 'T ';

(GDB) Where

#0 0x080482fd in core_test () at core_dump_test.c: 7

#1 0x08048317 in main () at core_dump_test.c: 12

#2 0x42015574in _ libc_start_main () from/lib/tls/libc. so.6

 

If you type where in GDB, you will see the stack information when the program crashes (the list of all called functions before the current function (including the current function), GDB only displays the last few functions ), we can easily find that our program called core_dump_test.c 7th line of code during the final crash, resulting in program crash. Note: Add option-G when compiling the program. You can also try other commands, such as fram and list. For more detailed usage, see the gdb documentation.

 

Location where the core file is created

Create in the current working directory of the process. It is usually in the same path as the program. However, if the chdir function is called in the program, the current working directory may be changed. The core file is created under the path specified by chdir. Many programs have crashed, but we cannot find the location of the core file. It is related to the chdir function. Of course, not all core files are generated when the program crashes.

When does the core file not be generated?

Core files are not generated under the following conditions:

(A) The process is set-user-ID, and the current user is not the owner of the program file;

(B) The process is set-group-ID, and the current user is not the group owner of the program file;

(C) The user has no permission to write the current working directory;

(D) The file is too large. Core File Permission (assuming that the file does not exist before) is usually User read/write, Group read and other reads.

Using GDB to debug core files, we are no longer helpless when a program crashes.