Detailed explanation of the composition of the CentOS Log System

Detailed explanation of the composition of the CentOS Log System

The log system consists of three parts:
1. What tools are used to record the logs generated by the system?
Two processes managed by the syslog service Script: syslogd and klogd to record the logs generated by the system;
The klogd process is dedicated to the log information service generated by the kernel.
The syslogd process is a log information service specially generated for the Application Service (except the kernel;
Only when the logger interface is called can the application call syslogd to help record log information.

2. As long as the system service is running, the log information will be generated. We use a dedicated tool to record it to the disk for convenience.

Analyze the log. The log file cannot be too large. We should delete it from the log file a long time ago. Who will accomplish these tasks?
Use the logrotate command to scroll the log files recorded by syslog.

3. to free people from busy work. Automated Management: rolling, backup, and compression of log files.
Use two loyal workers (processes): crond and anacron to complete all the above work.

Let's take a look at the implementation process of each of them:
1. syslog
Syslog is only a service script used to manage sysogd and klogd. Their configuration files are stored in/etc/syslog. conf.

[Root @ node2logs] # cat/etc/syslog. conf # Loganything (effectmail) oflevelinfoorhigher. # Don't 'tlogprivateauthenticationmessages! *. Info; mail. none; authpriv. none; cron. none/var/log/messages # asterisk (*) indicates all priority #*. info indicates to record all info-level logs of priority. Note that info-level logs are not included, and all logs with a higher info level are recorded. For example: notice, warn, err ,..., panic #*. info; mail. none; authpriv. none; cron. none indicates all priority of the asterisk (*) wildcard, but does not include mail, authpriv, cron. # indicates that the log information of the mail, authpriv, and cron systems will not be recorded in/var/log/messages. # Logallthemailmessagesinoneplace. mail. *-/var/log/maillog # The asterisk (*) indicates that all priority. # mail. * indicates that all levels of log information generated by the system mail are recorded. #-/Var/log/maillog horizontal line (-) indicates that the log information generated by System mail is asynchronously written to the disk. This means that it is first stored in the pagecache buffer pool. Then, fl it to the disk at a time based on a certain policy. # Mail is a very busy system and generates a large amount of log information. If the generated log information is immediately written to the disk, the efficiency is very low. The disk IO speed is very slow. The reason is that the system starts by default # two dirty pages are flushed to the thread. You can view [root @ node2 ~] through the/proc Interface # Cat/proc/sys/vm/nr_pdflush_threads. Linux will start more threads to write dirty pages based on the system's busy schedule. # Of course, we can also adjust this kernel parameter. However, it should be adjusted according to the number of hardware CPUs. # Everybodygetsemergencymessages *. emerg * # *. emerg * indicates that all system emerg-level logs are sent to all users. # Savebootmessagesalsotoboot. loglocal7. */var/log/boot. log # This record all levels of log information in the system boot process.

Configuration File Format:
Facility. priority/path
Facility is used to define who generates the log information: the log information generated during the running of the software and subsystem.
Priority is used to define what type of log information is recorded. All information generated by the application is recorded in logs.

File, or only the error log information of the application is recorded.
/Path/file_name is used to define where to write log information.

View common facility and priority:

[Root @ node2 ~] # Mansyslog. conf # common facilityThefacilityisoneofthefollowingkeywords: auth, authpriv, cron, daemon, kern, lpr, mail, mark, news, security (sameasauth), syslog, user, uucpandlocal0throughlocal7. # common metrics, inascendingorder: debug, info, notice, warning, warn (sameaswarning), err, error (sameaserr), crit, alert, emerg, panic (sameasemerg ).

Syslog has a uniform log file format for administrators to read.
Standard Format of Linux system log files:
The time when the event occurs. which host logs generate log information for the system events?
Fields are separated by spaces.

[Root @ node2 ~] # Tail/var/log/cronJun1210: 00: 01node2crond [5487] :( root) CMD (/usr/lib/sa/sa111) Jun1210: 01: 01node2crond [5490] :( root) CMD (run-parts/etc/cron. hourly)

Note:
Event Time: Jun 12 10:01:01
Log of the Host: node2
System that generates log information: crond [5490]
System Events: CMD (run-parts/etc/cron. hourly)

[Root @ node2 ~] # Tail-2/var/log/messagesJun1122: 13: 54node2syslogd1. 4.1: restart. Jun1122: 13: 54node2kernel: klogd1.4.1, logsource =/proc/kmsgstarted.

2. Log File scrolling:
All log files will increase rapidly over time and the number of visits.

Clean up to avoid unnecessary waste of disk space. In addition, this makes log analysis easier.
Logrotate, the log rolling handler, can automatically complete the log (of course, automatically rolling logs into periodic tasks ).

Compression, backup, deletion, and log mailing.
Each log file can be set to be processed every day, every week, or every month. It can also be set to scroll immediately when the log file is too large.
How logrotate works: defines the logrotate configuration file (/etc/logrotate. d/fine_name) for the rolling log file ),

Define the log scrolling event.
The log rolling handler reads the configuration file and then analyzes the log file based on the custom rolling policy.

File process scrolling.
The logrotate configuration file is divided into the main configuration file (/etc/logrotate. conf) and the sub-configuration file is defined in/etc/logrotate. d/

Directory (you need to use include to include this directory)
Defines the format of a rolling configuration file for a log file:
Log File full path {
Logrotate command
}
How does logrotate, a log rolling handler, roll the log file processes recorded by syslogd and kelogd?

[Root @ node2 ~] # Cat/etc/logrotate. conf # see "manlogrotate" fordetails # rotatelogfilesweekly defines the log clearing cycle. # The service that uses the "logrotate" command to scroll logs has been defined as a periodic task plan. # But it will be executed once every day at 04:02, # [root @ node2 ~] Ll/etc/cron. daily/logrotate #-rwxr-xr-x1rootroot180Feb272009/etc/cron. daily/logrotate # file/etc/cron. daily/logrotate is actually run # Row/usr/sbin/logrotate/etc/logrotate. conf # That is to say, at 04:02 every day, use the command [logrotate] to define it in/etc/logrotate. d directory # the log file of the application that meets the rolling conditions # process scrolling. Weekly # keep4weeksworthofbacklogs saves the log file rotate4 for the past four weeks # create an empty log file when rolling logs # createnew (empty) logfilesafterrotatingoldonescreate # uncommentthisifyouwantyourlogfilescompressed whether to use the compression function # compress # rpmpackagesdroplogrotationinformation=thisdirectory # Set/etc/logrotate. d directory. # If the configuration files in the/etc/logrotate. conf directory of the master configuration file and/etc/logrotate. d directory define the same commands #, the commands in the master configuration file will be overwritten. Otherwise, use the commands in the main configuration file. Include/etc/logrotate. d # nopackagesownwtmp -- we 'llrotatethemhere/var/log/wtmp {monthlyminsize1Mcreate0664rootutmprotate1}[Root @ node2 ~] # Cat/etc/logrotate. d/syslog #/var/log/messages,/var/log/secure,/var/log/maillog, #/var/log/spooler,/var/log/boot. log and/var/log/cron log files for log scrolling. /Var/log/messages/var/log/secure/var/log/maillog/var/log/spooler/var/log/boot. log/var/log/cron {# Call the general function sharedscripts for log scrolling # Run the command after the log is rolled, command postrotate/bin/kill-HUP between postrotate and endscript 'cat/var/run/syslogd. pid2>/dev/null' 2>/dev/null | true/bin/kill-HUP 'cat/var/run/rsyslogd. pid2>/dev/null' 2>/dev/null | trueendscript}

If we have written a log rolling configuration file, how can we know that the log rolling processing program

Is the operation successful.
You need to use the logrotate command to specify the configuration file process for testing.
[Logrotate] Command Format
Logrotate [Option] <configfile>
-D: displays the instruction execution process in detail to facilitate troubleshooting or understanding program execution.
-F: forcibly start the rolling operation of log files. Ignore the log rolling conditions defined in logrotate. Can be used to test the configuration

Is the log rolling correct.

Decide to use the logrotate application for log scrolling of the httpd server.
1) in the/etc/logrotate. d/directory, create the file httpd as the rolling configuration file of the log file of the httpd service.

[Root @ node2 ~] # Cat/etc/logrotate. d/httpd/usr/local/apache2/logs/access. log/usr/local/apache2/logs/error. log {missingok # If the log file is lost, create a new log file named ememptyrotate4size2m # Rolling sharedscriptspostrotate/bin/kill-1 'cat/var when the log file reaches 2 MB. /run/httpd. pid2>/dev/null' 2>/dev/nullendscript}

Note:
To use the-f parameter of the "logrotate" command to test the configured log rolling configuration file, remember to use the command

Rotate n to back up several rolling log files. If you do not use rotate
Command, [logrotate-f/etc/logrotate. d/configfile] will not back up the rolling logs.

When you run logrotate-f/etc/logrotate. d/configfile, you can see that the rolling log file is cleared.
If the-f option is used, an error is displayed in our configuration file. It is easy for us to modify.
You can also use [date] to adjust the system time to execute the/etc/cron. daily/logrotate task. The result is displayed.

2) simulate whether the logs of the httpd server are rolled when the application logrotate is executed.

[Root @ node2 ~] # Ll/usr/local/apache2/logs/total53172-rw-r -- r -- 1rootroot25702370Jun1116: 19access. log-rw-r -- 1rootroot833Jun1116: 18error. log

Use the-f option of the logrotate command to force the rolling of log files

[Root @ node2 ~] # Logrotate-f/etc/logrotate. d/httpd

Check whether the logs of the httpd server are rolling? Expectation or

[Root @ node2 ~] # Ll/usr/local/apache2/logs/total53176-rw-r -- r -- 1rootroot0Jun1116: 27access. log-rw-r -- 1rootroot25702370Jun1116: 19access. log.1-rw-r -- 1rootroot0Jun1116: 27error. log-rw-r -- 1rootroot833Jun1116: 27error. log.1

Note:
The results show that:
The "logrotate" command is used to scroll the logs of the httpd server.

3. How to Make the log scroll automatically?
In Linux, there are two Daemon Processes: crond and anacron, which are used to automatically execute system and user periodic tasks.
Crond and anacron are two completely different daemon processes used to regularly execute tasks.
Crond assumes that the server is running for 7x24 hours. This period of time will be omitted when the server is shut down for a period of time.

The crond task to be executed.
Anacron is used to execute tasks that are missed due to time disconsecutive crond.
In this way, they will not miss the scheduled task because of server shutdown or system time changes.
Periodic task plans are divided into the periodic task plans (defined in/etc/crontab) of the system and the periodic task plans of the user.

(Defined in/var/spool/cron/$ user ).
How does the crond daemon execute periodic task plans?
Crond will first read the periodic task plans of the system (defined in/etc/crontab) and the user's periodic task plans.

(Defined in/var/spool/cron/$ user), The crond daemon always monitors,
Whether the execution time of the periodic task plan has been reached. When the time and date in the task plan are the same as the current time and date of the system

The corresponding periodic task plan is executed. After the crond task is executed,
Any output will be sent to the owner of the scheduled periodic task as an email.

[Root @ node2 ~] # Cat/etc/crontabSHELL =/bin/bashPATH =/sbin:/bin:/usr/sbin: /usr/binMAILTO = rootHOME =/# run-parts01 *** rootrun-parts/etc/cron. hourly024 *** rootrun-parts/etc/cron. dailypolic** 0rootrun-parts/etc/cron. weekly1_1 ** rootrun-parts/etc/cron. monthly

Note:
This is the configuration file that defines the periodic Task Plan of the system.
02 4 * root run-parts/etc/cron. daily: run the command (SCRIPT) run-parts/etc/cron. daily at 04:02 every day.
/Etc/cron. daily is a command or script (run-parts) parameter.
The/etc/cron. daily directory defines many executable scripts to be executed. These scripts are the tasks to be executed.

What are the daily scheduled tasks?

[Root @ node2 ~] # Tree/etc/cron. daily/etc/cron. daily/| -- 0anacron | -- 0logwatch->/usr/share/logwatch/scripts/logwatch. pl | -- cups | -- logrotate ---------> logrotate log rolling task | -- makewhatis. cron | -- mlocate. cron | -- prelink | -- rpm '-- tmpwatch0directories, 9 files

View the script run-parts.

[Root @ node2 ~] # Cat/usr/bin/run-parts #! /Bin/bash # run-parts-concepttakenfromDebian # keepgoingwhensomethingfailsset + eif [$ #-lt1]; thenecho "Usage: run-parts <dir>" exit1fiif [! -D $1]; thenecho "Notadirectory: $1" exit1fi # Ignore *~ And *, scriptsforiin $1/* [^ ~,]; Do [-d $ I] & continue # Don 'trun *. {rpmsave, rpmorig, rpmnew, swp} scripts exclude some format files ["$ {I %. rpmsave }"! = "$ {I}"] & amp; continue ["$ {I %. rpmorig }"! = "$ {I}"] & amp; continue ["$ {I %. rpmnew }"! = "$ {I}"] & amp; continue ["$ {I %. swp }"! = "$ {I}"] & amp; continue ["$ {I %, v }"! = "$ {I}"] & amp; continue # execute files with executable permissions for/etc/cron. daily. If [-x $ I]; then $ i2> & 1 | awk-v "progname = $ I" \ 'progname {printprogname ": \ n" progname = "";} {print;} 'fidoneexit0 Description: The Script run-parts first checks the directory/etc/cron. daily file. If the conditions are met, execute it again. View the/etc/cron. daily/logrotate task script [root @ node2 ~] # Cat/etc/cron. daily/logrotate #! /Bin/sh/usr/sbin/logrotate/etc/logrotate. confEXITVALUE = $? If [$ EXITVALUE! = 0]; then/usr/bin/logger-tlogrotate "ALERTexitedabnormallywith [$ EXITVALUE]" fiexit0

Note:
The run-parts script first judges the/etc/cron. daily directory and then executes it if it meets the conditions.

View the/etc/cron. daily/logrotate task script

[Root @ node2 ~] # Cat/etc/cron. daily/logrotate #! /Bin/sh/usr/sbin/logrotate/etc/logrotate. confEXITVALUE = $? If [$ EXITVALUE! = 0]; then/usr/bin/logger-tlogrotate "ALERTexitedabnormallywith [$ EXITVALUE]" fiexit0

Note:
/Usr/sbin/logrotate/etc/logrotate. conf
The logrotate log rolling program performs operations such as log scrolling, backup, and Compression Based on the configfile (/etc/logrotate. conf) configuration file.