Analysis and troubleshooting of system faults in Linux

When dealing with a variety of failures in a Linux system, the symptoms of the failure are the first to be discovered, and the cause of this failure is the key to eventual troubleshooting. Familiar with the Linux system log management, understanding common fault analysis and solutions, will help administrators quickly locate the point of failure. "The right remedy" in time to solve a variety of system problems.

1, log analysis and management

Log files are files used to record various running messages in a Linux system, equivalent to the "diary" of a Linux host. Different log files record different types of information, such as: Linux kernel message, user login record, program error, etc. Log files are useful for diagnosing and resolving problems in the system, because programs that run on a Linux system usually write system messages and error messages to the appropriate log files, so that the system will be "well documented" if problems arise. In addition, when the host is under attack, the log file can also help look for traces left by the attacker. Below I will introduce the main log and analysis management methods in Linux system.

1.1, the main log files include the following three kinds of types:

> Kernel and System log: This log data is managed uniformly by the system service Syslog, which determines where kernel messages and various system program messages are recorded, based on the settings in its primary configuration file "/etc/syslog.conf". There are a number of programs in the system that have their own log files managed by the syslog, so that the log records used by these programs also have similar formatting.

> User log: This log data is used to record information about the Linux system user login and exit system, including user name, login terminal, logon time, source host, process operation in use, etc.

> Program log: Some applications will choose to manage a log file independently (rather than to the Syslog service management) to record various event information during the running of the program. Because these programs are only responsible for managing their own log files, there may be significant differences in the logging format used by different programs.

The Linux system itself and the log files of most server programs are placed by default in the directory "/var/log". Some programs share a log file, some programs use a single log file, and some large server programs because the log file is not one, so will be in the "/var/log/" directory to create a corresponding subdirectory to store log files, so as to ensure that the log file directory structure is clear, You can also quickly locate log files. There are quite a few log files that only root users have permission to read, which guarantees the security of related log information.

>>>>>>>>: List view the various log files and subdirectories in the "/var/log" directory.

For some common log files in Linux systems, it is necessary to familiarize themselves with their corresponding uses, so as to find the problem faster when needed, and solve all kinds of faults in time. Such as:

>/var/log/messages: Records the Linux kernel messages and the public log information for various applications, including startup, IO errors, network errors, program failures, and so on. For applications or services that do not use a stand-alone log file, you can generally obtain relevant event logging information from the file.
>/var/log/cron: Records event messages generated by Crond scheduled tasks.
>/VARLOG/DMESG: Records the various event information of the Linux system during the boot process.
>/var/log/maillog: Records e-mail activities that enter or issue a system.
>/var/log/lastlog: Recent successful logon events and last unsuccessful logon events.
>/var/log/rpmpkgs: Records the system to install each RPM package list information.
>/var/log/secure: Record The event information in the process of user login authentication.
>/var/log/wtmp: Log logon, logoff, and system startup and downtime events for each user.
>/var/log/utmp: Log details for each user who is currently logged on

1.2. log file Analysis

Familiar with the main log in the system, we do the analysis of the log file to understand. The purpose of the analysis log file is to find the key information by browsing the log, to debug the system service, to judge the cause of the malfunction. This is mainly about the basic format and analysis method of three kinds of log files.

This column more highlights: http://www.bianceng.cn/OS/Linux/

For most text-formatted log formats (such as kernel and system logs, most program logs), you can view the contents of the log as long as you use text processing tools such as tail, more, less, and cat. For some binary-formatted log files (eg: User logs), you need to use the appropriate query commands.

1, Kernel and system log:

The kernel and system log features are primarily provided by the default installed syslogd-1.4.1-39.2 package, which installs KLOGD, syslogd two programs, and is controlled through the Syslog service to record system kernel messages and various application messages, respectively. The configuration file used by the Syslog service is "/etc/syslog.conf".

Typically, the kernel and most system messages are recorded in the common log file "/var/log/messages", while other program messages are recorded in different files, log messages can be logged to a particular storage device, or sent directly to the user.

>>>>> View the contents of the Log configuration file "/etc/syslog.conf"