Linux system log and log analysis

Linux system log and log analysis The Linux system has a very flexible and powerful logging function, which can save almost all the operation records and retrieve the information we need.

The default log daemon for most Linux distributions is syslog, located in/etc/syslog or/ETC/SYSLOGD, and the default profile is/etc/syslog.conf, and any program that wants to generate the log can send information to the syslog.

Linux system cores and many programs generate various error messages, warning messages, and other prompts that are useful for administrators to understand the state of the system, so they should be written to a log file. The procedure to complete this process is syslog. Syslog can save logs to different files based on the category and priority of the log. For example, for easy reference, you can separate the kernel information from other information and save it to a separate log file. In the default configuration, log files are usually saved in the "/var/log" directory. Log types The following are common log types, but not all Linux distributions contain these types:

type	Description
Auth	Logs generated when the user authenticates, such as the login command, the SU command.
Authpriv	Similar to auth, but can only be viewed by specific users.
Console	Messages for the system console.
Cron	The log that is generated when the system periodically executes scheduled tasks.
Daemon	Logs generated by some daemons.
Ftp	FTP Service.
Kern	System kernel messages.
Local0.local7	Used by custom programs.
Lpr	Related to printer activity.
Mail	Mail logs.
Mark	Generates a timestamp. The system outputs the current time to the log file at intervals, and each line is formatted like May 11:17:09 RS2-MARK-to infer the approximate time the system has failed.
News	The message generated by the Network News Transfer Protocol (NNTP).
Ntp	The message generated by the Network Time Protocol (NTP).
User	The user process.
Uucp	UUCP subsystem.

Log priority is a common log priority, see subscript:

Priority Level	Description
Emerg	In an emergency, the system is not available (such as a system crash) and is generally notified to all users.
Alert	Requires immediate repair, such as system database corruption.
Crit	Dangerous situations, such as hard drive errors, can hinder some of the program's functionality.
Err	Generic error message.
Warning	Warning.
Notice	Not an error, but may need to be handled.
Info	Generic messages are generally used to provide useful information.
Debug	The information that is generated by the debug program.
None	No priority, no log messages are logged.

Common log files All system applications create log files under the/var/log directory, or create subdirectories and create log files. For example:

Files/Directories	Description
/var/log/boot.log	Turn on or restart the log.
/var/log/cron	Scheduling task logs
/var/log/maillog	Mail logs.
/var/log/messages	This log file is a summary of many process log files, from which you can see any intrusion attempts or successful intrusions.
/VAR/LOG/HTTPD Directory	Apache HTTP Service log.
/var/log/samba Directory	Samba software Log

The/etc/syslog.conf file/etc/syslog.conf is a syslog configuration file that determines where to save the log based on the log type and priority. The typical syslog.conf file format is as follows:

*.err;kern.debug;auth.notice/dev/consoledaemon,auth.notice           /var/log/messageslpr.info                     /var/log/ lpr.logmail.*                       /var/log/mail.logftp.*                        /var/log/ftp.logauth.*                       @see. xidian.edu.cnauth.*                       Root, Amroodnetinfo.err                  /var/log/netinfo.loginstall.*                    /var/log/install.log*.emerg                      **.alert                      |PROGRAM_ namemark.*                       /dev/console

The first column is a combination of log type and log priority, each combination of type and priority is called a selector, followed by a file that holds the log, a server, or a terminal for the output log. The syslog process determines how the log is manipulated based on the selector.

A few notes about the configuration file:

• Log type and priority by dot number (.) partitions, such as Kern.debug, represent debug information generated by the kernel.
• Kern.debug priority is greater than debug.
• An asterisk (*) indicates all, such as *.debug, which represents all types of debug information, and kern.* represents all messages generated by the kernel.
• You can use commas (,) to separate multiple log types, and separate multiple selectors with semicolons (;).

The operations on the log include:

• Output the log to a file, such as/var/log/maillog or/dev/console.
• The message is sent to the user, and multiple users are separated by commas (,), such as Root, Amrood.
• Send a message to a user program through a pipeline, noting that the program is placed in a pipe (|) character Behind.
• Sends a message to a syslog process on another host, when the/etc/syslog.conf file is followed by a host name that begins with @, such as @see.xidian.edu.cn.

The Logger command logger is a shell command that allows you to use Syslog system log modules and write a line of information directly from the command line to the system log file.

The syntax for the Logger command is:

Logger [-i] [-f filename] [-p priority] [-t tag] [message ...]

Each option has the following meanings:

Options	Description
-F filename	Use the contents of the filename file as a log.
-I.	Each row records the ID of the logger process.
-P Priority	Priority must be a full selector of the shape such as facility.priority, with the default priority of User.notice.
-T Tag	Marks each record line with the specified label.
Message	Log content to be written, multiple logs are separated by spaces, and standard input is used as the log content if no log content is specified and the-F filename option is blank.

For example, write the results of the ping command to a log:

$ ping 192.168.0.1 | Logger-it logger_test-p local3.notice&$ tail-f/var/log/userlogoct 6 12:48:43 kevein logger_test[22484]: PING 192.1 68.0.1 (192.168.0.1) bytes of data. OCT 6 12:48:43 kevein logger_test[22484]: bytes from 192.168.0.1:icmp_seq=1 ttl=253 time=49.7 msoct 6 12:48:44 Kevein  logger_test[22484]: bytes from 192.168.0.1:icmp_seq=2 ttl=253 time=68.4 msoct 6 12:48:45 Kevein logger_test[22484]: 64 Bytes from 192.168.0.1:icmp_seq=3 ttl=253 time=315 msoct 6 12:48:46 Kevein logger_test[22484]: + bytes from 192.168.0.1 : icmp_seq=4 ttl=253 time=279 msoct 6 12:48:47 kevein logger_test[22484]: bytes from 192.168.0.1:icmp_seq=5 ttl=253 ti  me=347 msoct 6 12:48:49 kevein logger_test[22484]: bytes from 192.168.0.1:icmp_seq=6 ttl=253 time=701 msOct 6 12:48:50 Kevein logger_test[22484]: bytes from 192.168.0.1:icmp_seq=7 ttl=253 time=591 msoct 6 12:48:51 Kevein logger_test[224 +]: bytes from 192.168.0.1:icmp_seq=8 ttl=253 time=592 msoct 6 12:48:52 Kevein logger_test[22484]: bytes from 192.168.0.1:icmp_seq=9 ttl=253 time=611 msoct 6 12:48:53 Kevein logger_test[22484]: 64 Bytes from 192.168.0.1:icmp_seq=10 ttl=253 time=931 ms

The result of the ping command was successfully output to the/var/log/userlog file.

Command logger-it logger_test-p local3.notice the meaning of each option:

• -I: The process ID is recorded on each line;
• -T logger_test: Each line of records is labeled "Logger_test";
• -P Local3.notice: Sets the log type and priority.

Log dump log dumps are also called log back volumes or log rotations. Logs in Linux typically grow quickly, consume a lot of hard disk space, and need to be stored separately when the log file reaches the specified size.

Syslog is only responsible for receiving logs and saving to the appropriate files, but will not manage the log files, so often cause log files too large, especially the Web server, can easily exceed 1G, to the search difficulties.

Most Linux distributions use Logrotate or Newsyslog to manage logs. Logrotate program can not only compress log files, reduce storage space, but also can send logs to the designated e-mail, convenient for administrators to view the log in time.

For example, to specify that the mail log/var/log/maillog be dumped at more than 1G, once a week, the logrotate process checks the size of the/var/log/maillog file every other week:

• If there is no more than 1G, no action is made.
• If between 1g~2g, a new file/var/log/maillog.1 is created and the extra 1G log is transferred to the file to slim down the/var/log/maillog file.
• If between 2g~3g, the new file/var/log/maillog.2 will continue to be created, and the contents of/var/log/maillog.1 will be transferred to the file, and the contents of the/var/log/maillog will be transferred to/var/log/ Maillog.1 to keep the/var/log/maillog file no more than 1G.

As you can see, each dump creates a new file (if it doesn't exist), and the name format is the log file name plus a number (auto-grow from 1) to keep the current log file and the post-dump log file not exceeding the specified size.

Logrotate's main profile is that the/ETC/LOGROTATE.CONF,/ETC/LOGROTATE.D directory is a supplement to/etc/logrotate.conf, or in order not to make/etc/logrotate.conf Too large to set.

You can view its contents through the Cat command:

$cat/etc/logrotate.conf# See ' Man logrotate ' for details  //can view Help document # Rotate log files weeklyweekly                             //Set weekly Dump # k EEP 4 weeks worth of backlogsrotate 4                           //MAX Dump 4 times # Create new (empty) log files after rotating old onescreate                             //when dump The piece is not stored when it is created # uncomment this if you want your log files compressed#compress                          //Compressed Dump # RPM Packages drop log Rotation info Rmation           into this DIRECTORYINCLUDE/ETC/LOGROTATE.D//other log files are dumped in this directory # no packages own wtmp--we'll rotate them h ere/var/log/wtmp {                    //settings/var/log/wtmp log file dump Parameters    monthly                        //monthly dump    create 0664 root utmp          // When the file does not exist after the dump, it is created, the file owner is root, the owning group is utmp, the corresponding permission is 0664    rotate 1                       ///dump at most once}

Note: Include allows administrators to centralize multiple scattered files into one, similar to the C language #include, to include the contents of other files in the current file.

Include is useful in that some programs place the dump log's configuration file in the/ETC/LOGROTATE.D directory, which overrides or adds/etc/logrotate.conf configuration items, and if no relevant configuration is specified, the/etc/ The default configuration for logrotate.conf.

Therefore, it is recommended to use/etc/logrotate.conf as the default profile and a third-party program to customize the configuration file in the/ETC/LOGROTATE.D directory.

Logrotate can also be run directly as a command to modify the configuration file.

Linux system log and log analysis