Analysis and solution of the crash of Linux system

First, the cause of the common crash

Second, log analysis

The log system, which is controlled by the Rsyslog.service service, is used to record the system kernel and the log information of each application respectively. Configuration file/etc/rsyslog.conf

/var/log/messages records system kernel messages and common log information for various applications, including startup, IO error, network error, program error, etc., for applications or services that do not use stand-alone log files, you can generally get logging information about events from that file.     /var/log/cron records the event messages generated by Crond Scheduled Tasks/VAR/LOG/DMESG record the various event information that the system has during the boot process/var/log/maillog record incoming or emitted system e-mail activity/var/log/lastlog Recent successful logon times and last unsuccessful logon events/var/log/rpmpkgs record system install each RPM package list information/var/log/secure record user logon authentication process event information/var/log/wtmp log each user login , logout and system boot and Shutdown event information, can not be cat view, only LAST/LASTB view

Typically, the kernel and most system messages are recorded in the public log file/var/log/messages, while other program messages are recorded in different log files, and log messages can be recorded on a particular storage device or sent directly to the user.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/8B/8A/wKiom1hQvCygA1ADAABYMaSMke4903.png "title=" Rsyslog.png "alt=" Wkiom1hqvcyga1adaabymasmke4903.png "/>

2.2 Importance of log messages

Linux system based on the importance of log messages, divided into different priority levels (the smaller the number, the higher the priority, the more important the message)

>0 Emerg (Emergency): a condition that causes the host system to be unavailable. >1 Alert (warning): A problem that must be taken immediately to resolve. >2 crit (severe): a more serious situation. >3 ERR (Error): An error occurred while running. >4 WARNING (Reminder): May affect the system function, need to remind users of important events. >5 NOTICE (Note): Events that do not affect normal functionality, but need attention. >6 Info: General information. >7 bebug (Debug): Program or system debugging information.

2.3 Log Record format

Most log files that are managed uniformly by the Rsyslog service are basically the same in the logging format. For example, public log files/var/log/messages the basic format of the kernel and system log

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M00/8B/8E/wKiom1hQ9LSDbLnLAACTFIoOhb8658.png "title=" Qq20161214113820.png "alt=" wkiom1hq9lsdblnlaactfioohb8658.png "/> Log each line represents a message, each message is made up of four fields in a fixed format

: Time Label: Date and time when the message was issued.   Host Name: The name of the computer that generated the message.   : Subsystem Name: The name of the application that issued the message. Message: The exact content of the message.

Log information is generally sent to other device stores, even if the hacker wants to clear the traces after the intrusion, so that the administrator is difficult to find.

2.4 User logs

Wtmp/utmp/lastlog, etc. are user log files, save the system user login, exit and other related information, are binary files, can not be directly tail/cat view, you need to who/w//last/lastb/ac and other commands to query

2.5 Log File Management policy

Log backup and archive: Log files are also important data and need to be backed up and archived. : Extended Log Retention period: Log data should be kept as long as possible in the event of a rich storage space. : Control log access: The log data may contain various kinds of sensitive information, such as: Account number, password, etc. Therefore, it is necessary to strictly control its access rights. Centralized management log: Use a centralized log server to manage log records sent by each server. The advantage is to facilitate the collection, collation and analysis of the log, to eliminate accidental loss, malicious tampering or deletion.

2.6 Centralized Log Management

2.6.1 Log Server A

/etc/sysconfig/rsyslog Change the contents of the syslogd_options variable to "-r-x-M 0".

-r means allow logging from other servers to be accepted.

-X indicates that DNS domain name resolution is not performed

-M indicates the time interval for logging, and 0 means that the feature is disabled

2.6.2 Client B

Modifying the/etc/rsyslog.conf configuration file

vi/etc/syslog.conf cron.* @173.17.17.3 Service syslog restart

2.6.3 Verification

Modify the contents of Crontab on B and observe if the log is recorded on a.

Analysis and solution of the crash of Linux system