Simple and Easy-to-use log management tool Logrotate

Simple and Easy-to-use log management tool Logrotate
Preface The log is like the life recorder of a program, recording the details of the program running. Careful selection of logs to record: you do not want to experience the logs that really record problems in the vast sea of logs; careful regular compression of transfer logs: When a fault occurs, the logs are lost, at this time, any excuse is pale and powerless. To record the logs, You need to carefully compile the application. This article describes how to use the logrotate program that comes with Linux to carefully organize our lovely log files. The introduction log is actually the document in this article, and it is a document with increasing content. Processing is usually performed by day or by size to back up, compress, or dump, which is very simple. /Var/log/messages is a familiar location for storing system logs. If the content of the file grows rapidly and may fill the entire hard disk in a few days, it is particularly important to back up, compress, or dump the file. What should I do manually? It may be: Write a log processing script to check that if the log file size exceeds 10 MB, copy a log and compress the dump, and clear the original log in the middle; the Crontab uses this script as a scheduled task and runs it every day. Logrotate is actually a tool that acts as a script. It allows users to configure rules to detect and process log files. With Cron, processing can be scheduled. Logrotate pre-fabricated a large number of judgment conditions and processing methods, which can greatly reduce the burden on handwritten scripts and the possibility of errors. Logrorate checks log file attributes, compared with the detected conditions configured by the user, the system can handle the conditions that meet the conditions according to the user configuration requirements. The entire process can be scheduled through Cron, which is actually a very classic Linux solution, it can be easy to use. The following are the key points for logrotate operation:

/Usr/bin/logrotate program location;/etc/cron. by default, daily/logrotate enables Cron to execute logrotate once every day;/etc/logrotate. conf global configuration file;/etc/logrotate. d. Store the configuration file directory of the application to overwrite the global configuration. The global configuration file/etc/logrotate. conf # cat/etc/logrotate. conf # see "man logrotate" for details # rotate log files weeklyweekly # keep 4 weeks worth of backlogsrotate 4 # create new (empty) log files after rotating old onescreate # use date as a suffix of the rotated file # dateext # uncomment this if you want your log files compressed # compress # RPM packages drop log rotation information into this directoryinclude /etc/logrotate. d # no packages own wtmp and btmp -- we'll rotate them here/var/log/wtmp {monthly create 0664 root utmp minsize 1 M rotate 1}/var/log/btmp {missingok monthly create 0600 root utmp rotate 1} # system-specific logs may be also be configured here.

 

View the global configuration file cat/etc/logrotate. conf, you can find that # Is a line comment, and the configuration items appear in the form of keywords (to know the role of each keyword, the simplest is to view it directly by man ). Taking a closer look at the comments of each keyword, it is not difficult to find that, for example: weekly: indicates that logs are processed every week; rotate4: a maximum of four rotation backups are maintained. This article will detail the rotation itself, which is very interesting; create: After the log file is processed, a new log file is generated, of course, the same permission as possible; dateext: No timestamp by default; compress: no compression by default; the wtmp and btmp logs are processed separately, so you realize that the separate configuration can be placed in/etc/logrotate. d directory, or directly put it in the global configuration. Therefore, when the logrotate program is executed, logrotate is literally used to process logs every week by default. logrotate retains a maximum of four logs for rotation. The processing method is no compression or timestamp, after processing, a file with the same name is generated. Of course, these are the default settings, and separate requirements for wtmp and btmp log processing and there are a lot of processing requirements in the include/etc/logrotate. d directory. The configuration files in this directory are not read and executed one by one after logrotate is executed. Custom configuration file storage directory/etc/logrotate. d/# ls/etc/logrotate. d/cups debug dracut httpd iptraf mysqld ppp psacct sssd subpartition-manager syslog tomcat6 up2date vsftpd wpa_supplicant yum for example, view the content of the sssd configuration file:

# cat /etc/logrotate.d/sssd/var/log/sssd/*.log {    weekly    missingok    notifempty    sharedscripts    rotate 2    compress    postrotate        /bin/kill -HUP `cat /var/run/sssd.pid  2>/dev/null`  2> /dev/null || true    endscript}

 

The basic format and global configuration file/etc/logrotate can be found. conf is the same. It is hard to imagine that a single log configuration requires a higher priority. If the configuration of the same project appears in the global configuration, the configuration of the same project must completely overwrite the global one. Cron execution: the/etc/cron. daily/logrotate configuration file is specified, and logrotate can be executed independently. Of course, cron can also be used for scheduled execution;

# cat /etc/cron.daily/logrotate#!/bin/sh/usr/sbin/logrotate /etc/logrotate.conf >/dev/null 2>&1EXITVALUE=$?if [ $EXITVALUE != 0 ]; then    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"fiexit 0

 

The default logrotate has been placed in/etc/cron. the daily/logrotate directory clearly enables cron to execute the logrotate program once a day. Of course, you can also put the script at other times, such as every minute, you can even write a crontab expression to allow logrotate to specify the configuration file and time for execution. Now, we have read the global configuration file of logrotate, which is a separate configuration file, how to Use cront for scheduled execution. To enhance the memory, we should first summarize: logrotate is a program dedicated to processing log files; processing requires the user to configure rules, for example, specify what action to do if it exceeds 10 MB; rules can be configured in an independent configuration file. Of course, there is a global default configuration file. You need to know where each configuration file is stored. logrotate is executed once, all configuration files are searched to process logs as required. You can use cron to perform logrotate on a regular basis. People who have tested man logrotate probably know how to use logrotate. Common options are as follows:

#1. the debug (d = debug) parameter is the configuration file. If this parameter is not specified, the global configuration file logrotate-d/etc/logrotate is executed. d/test. conf #2. forced Execution (-f = force) can be used with-v (-v = verbose). Note that the debugging information is carried by-v by default; logrotate-v-f/etc/logrotate. d/test. in this example, the User-Defined configuration file is used to compress the specified log file to test the use of logrotate. Note that logrotate must be executed by the root user. (However, you can use configuration items to specify the generated log files as common users ). #1. generate a log file man ps> test. logll-h test. log-rw-r -- 1 root 54 K Sep 6 11: 36 test. log #2. compile the logrotate configuration file cat/etc/logrotate. d/test. conf/var/log/test. log {compressrotate 4 size 30 kcreate 0600 root} #3. debug whether the compressed file logrotate-d/etc/logrotate can be generated according to the configuration file requirements. d/test. confreading config file/etc/logrotate. d/test. confreading config info for/var/log/test. logHandling 1 logsrotating pattern:/var/log/test. log 30720 bytes (4 rotations) empty log files are rotated, old logs are removedconsidering log/var/log/test. log needs rotatingrotating log/var/log/test. log, log-> rotateCount is 4 dateext suffix '-20150906 'glob pattern'-[0-9] [0-9] [0-9] [0-9] [0-9]] [0-9] [0-9] [0-9] 'renaming/var/log/test.log.4.gz to/var/log/test.log.5.gz (rotatecount 4, logstart 1, I 4), renaming/var/log/test.log.3.gz to/var/log/test.log.4.gz (rotatecount 4, logstart 1, I 3 ), renaming/var/log/test.log.2.gz to/var/log/test.log.3.gz (rotatecount 4, logstart 1, I 2 ), renaming/var/log/test.log.1.gz to/var/log/test.log.2.gz (rotatecount 4, logstart 1, I 1 ), renaming/var/log/test.log.0.gz to/var/log/test.log.1.gz (rotatecount 4, logstart 1, I 0), renaming/var/log/test. log to/var/log/test. log.1creating new/var/log/test. log mode = 0600 uid = 0 gid = 0 compressing log with:/bin/gzipremoving old log/var/log/test.log.5.gz error: error opening/var/log/test.log.5.gz: no such file or directory #4. the debugging result is normal. In the actual test, logrotate-f/etc/logrotate. d/test. confll-h test. log *-rw ------- 1 root 0 Sep 6 11: 44 test. log-rw-r -- 1 root 14 K Sep 6 11: 44 test.log.1.gz

 

The test is normal. The preceding logrotate-f/etc/logrotate. d/test. conf command can be fully written into crontab and executed according to the required time. From the above debug information, we can extract the rotation part of the log to understand what rotation is.

rotating log /var/log/test.log, log->rotateCount is 4renaming /var/log/test.log.4.gz to /var/log/test.log.5.gz (rotatecount 4, logstart 1, i 4),renaming /var/log/test.log.3.gz to /var/log/test.log.4.gz (rotatecount 4, logstart 1, i 3),renaming /var/log/test.log.2.gz to /var/log/test.log.3.gz (rotatecount 4, logstart 1, i 2),renaming /var/log/test.log.1.gz to /var/log/test.log.2.gz (rotatecount 4, logstart 1, i 1),renaming /var/log/test.log.0.gz to /var/log/test.log.1.gz (rotatecount 4, logstart 1, i 0),renaming /var/log/test.log to /var/log/test.log.1emoving old log /var/log/test.log.5.gzerror: error opening /var/log/test.log.5.gz: No such file or directory

 

According to the configuration file requirements, rotate four copies. The preceding log is abbreviated as "rotateCount = 4mv 4 5mv 3 4mv 2 3mv 1 2rm 5" in pseudocode, which is easy to understand, it is a constant rename like a second-level system to the right shift. After figuring out the operation mechanism of logrotate and testing it in the field, we will use multiple examples to quickly configure it. The function of clearing but not deleting the log file copytruncate is to first copy the current log file for processing, and then clear the source log file so that it can continue to receive logs. Of course, there may be several gaps between replication and clearing.

$ cat logrotate.conf /tmp/output.log {            size 1k            copytruncate            create 700 bala bala            rotate 4            compress }

After processing logs, execute the custom scripts postrotate and endscript to write custom scripts for processing logs or other definitions, which is highly scalable; for example, because logrotate can only specify a timestamp for compressed logs to days, you can customize the names of files in the script;

$ cat logrotate.conf /tmp/output.log {            size 1k            copytruncate            rotate 4            compress            postrotate                       /home/bala/myscript.sh            endscript }

 

Change the compression program to use .gz. Of course, you can customize the compression program and suffix;

$ cat logrotate.conf /tmp/output.log {            size 1k            copytruncate            create            compress            compresscmd /bin/bzip2            compressext .bz2            rotate 4 }