Linux Panic log Analysis _unix Linux

System crashes, generally divided into two situations: first, the hardware problem, the second is the software problem.


First, the hardware problem


The following points can be considered for analysis:


1, do not overclocking the CPU, if already overclocking first revert to the original frequency


Although there is no problem with running normally, there may be unexpected failures in the use of high load. In particular, Linux systems in some applications, can be the performance of the hardware to the limit, but such hardware running Windows may be no problem.


2, confirm the power supply sufficient


You must ensure that the power supply meets the load in a high load state.


3. Check memory status with Memtest86


4, restore the BIOS to the default state


For servers, you can use your own monitoring tools to test, but also a good debugging method.


Second, the software problem


If the hardware problem has been basically eliminated, then we must consider the system information from the software to get the panic state.

1, if the luck is good enough, the system crash is not necessarily completely dead (at this point the keyboard may also be able to respond), then we can use SYSRQ Dafa.


The premise is that we must first turn on the SYSRQ function:


#echo "1" >/proc/sys/kernel/sysrq

#setterm-blank


In this way, when there is a problem with the system, we can use:


Reference

ALT+SYSRQ-T Get process System stack information

ALT+SYSRQ-M Get memory allocation information

ALT+SYSRQ-W Get current Register information

More Hotkeys can refer to the/usr/src/linux/documentaion/sysrq.txt on the system


Among them, Setterm-blank can turn off the characters under the black screen protection, easy to record screen information.


2, in order to let the screen display more kernel debugging information, you can modify the console display mode for 80x25, in the/boot/grub/menu.lst of the corresponding kernel line last, add vga=0x305, such as:


Reference

KERNEL/BOOT/VMLINUZ-2.4.21-9.30AXSMP ro ROOT=/LABEL=/1 vga=0x305


3, if the keyboard is also unfortunate died, then we can only use the serial port, the system information sent to another system, methods:


Modify the/boot/grub/menu.lst file to add the core parameter "Console=ttys0 console=tty1" to the kernel line, such as:


Reference

KERNEL/BOOT/GRUB/VMLINUZ-2.4.21-9.30AXSMP ro root=/label=/1 console=ttys0 console=tty1


Then, modify the/etc/sysconfig/syslog and add the KLOGD option "-C 7", such as:


Reference

klogd_options= "-x-c 7"


Reboot the server, and test:

1 Connect the client and the server using the serial port direct connection, run on the client:


Cat/dev/ttys0


Run on the server:


echo Hi >/dev/ttys0


If the client has "HI" output.

2) running on the server:


echo W >/proc/sysrq-trigger


See if the client has the appropriate kernel information output

3) running on the server:


Modprobe Loop


See if the client has the appropriate kernel information output


If the test is passed, run on the client:


CAT/DEV/TTYS0 | Tee/tmp/result


When the panic occurs, we can see the required kernel information from the client (see/tmp/result).


Third, summary


The general causes of Linux system crashes are:

System hardware issues (SCSI card, motherboard, RAID card, HBA card, NIC, hard drive, etc.)

Peripheral Hardware issues (network, etc.)

Software issues (System, application software)

Drive Bugs (Find a new driver)

Core system bug (go to lkml to see, or replace core to try again)

System settings (revert to default state, firewall shutdown, etc.)