Linux Log timing polling process in detail (logrotate)

Logrotate Introduction

Log files are an extremely important tool for Linux system security. Log files contain useful information about the events that occur in the system, which are often used during troubleshooting or during system performance analysis. When the log files grow, it is necessary to cut the time, otherwise, the speed and performance of writing logs will also decline, more inconvenient for us to archive, query.

So with the use of Logrotate, logrotate is a very useful tool that can automatically truncate (or round) the log, compress, and delete old log files. For example, you can set Logrotate to have/var/log/foo log files every 30 days and delete logs for more than 6 months. Once configured, the logrotate is fully automated, without any further human intervention.

Logrotate Configuration file Location

The Linux system installs the Logrotate tool by default and it defaults to the configuration file in:
/etc/logrotate.conf
/etc/logrotate.d/

Logrotate.conf is the main configuration file, LOGROTATE.D is a directory, all the files in the directory are actively read into the/etc/logrotate.conf execution.

In addition, if there are no details in the file in/etc/logrotate.d/, the default value will be set to/etc/logrotate.conf the file.

Logrotate will invoke the configuration file/etc/logrotate.conf when it is actually running.

A custom configuration file can be placed in the/ETC/LOGROTATE.D directory to overwrite the default values of the Logrotate.

Timing round robin mechanism

Logrotate is based on Cron, and its script is/etc/cron.daily/logrotate, and the log rotation is done automatically by the system.

Logrotate This task is placed by default on Cron's daily scheduled task cron.daily/etc/cron.daily/logrotate
/etc/directory There are also cron.weekly/, cron.hourly/, cron.monthly/directories can be placed on a scheduled task

[/etc]$ cat/etc/cron.daily/logrotate

#!/bin/sh

# clean Non existent log file entries from status file

Cd/var/lib/logrotate

TEST-E Status | | Touch status

Head-1 Status > Status.clean

Sed ' s/'//g ' status | While read logfile date

Do

[-E "$logfile"] && echo "\" $logfile \ "$date"

Done >> Status.clean

MV Status.clean Status

Test-x/usr/sbin/logrotate | | Exit 0

/usr/sbin/logrotate/etc/logrotate.conf

The last line of the actual polling command here
/usr/sbin/logrotate/etc/logrotate.conf

Define the daily task of the script cron.daily/logrotate, and then view the contents of crontab, which set the corresponding cron.xxly execution time

[/etc]$ Vim/etc/crontab

Shell=/bin/sh

Path=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m H Dom Mon Dow user command

* * * * * Root CD/&& run-parts--report/etc/cron.hourly

6 * * * Root test-x/usr/sbin/anacron | | (CD/&& run-parts--report/etc/cron.daily)

6 * * 7 root test-x/usr/sbin/anacron | | (CD/&& run-parts--report/etc/cron.weekly)

6 1 * * Root test-x/usr/sbin/anacron | | (CD/&& run-parts--report/etc/cron.monthly)

As you can see, as long as it's in

• /etc/cron.daily/the following tasks are performed every day 6:25

• /etc/cron.weekly/the following tasks are performed every Sunday 6:47

• /etc/cron.monthly/the following tasks are performed at 1th # 6:52 per month

If you can't wait for cron to automatically perform log rotation, to manually force the cutting log, you need to add-f parameter;

However, it is best to verify the (-D parameter) with the debug option before it is formally executed, which is also important for debugging

#/usr/sbin/logrotate-f/etc/logrotate.d/nginx//not to time or not to cutting conditions, forced cutting

#/usr/sbin/logrotate-d-f/etc/logrotate.d/nginx//Output cutting debug information

So far, we know how Logrotate realizes automatic cutting logs.

Logrotate Configuration Cases

Nginx Common Log Cutting configuration

/data/log/nginx/*.log/data/log/nginx/*/*.log {# to cut a log file on a match

Weekly # Weekly Cut

Missingok # During the log rotation, any errors will be ignored, such as "files cannot be found".

Rotate 6 # Keep 6 backups

Compress # compression

Delaycompress # delaycompress and compress are compressed when the dump log file is used for the next dump

Notifempty # If it's an empty file, don't dump it.

Create 0644 www-data ymserver # mode owner group dump file, creating a new log file with the specified file mode

Sharedscripts # Here's the details.

Prerotate # Instructions that need to be executed before logrotate dumps, such as modifying the properties of a file;

if [-d/etc/logrotate.d/httpd-prerotate]; Then \

Run-parts/etc/logrotate.d/httpd-prerotate; \

FI \

Endscript

Postrotate # Instructions that need to be executed after the logrotate dump, such as restarting (KILL-HUP) a service! Must be independent

[-s/run/nginx.pid] && kill-usr1 ' cat/run/nginx.pid '

Endscript

Su root Ymserver # rotation log when switching settings for user/user group to execute (default is root), if the set User/group does not have permission to allow the file to be specified by the Create option, the error will be triggered.

}

If you want to configure a 0-point daily cutting task, how do you do it? Our logrotate default daily execution time has been written down to the/etc/cron.daily/directory, and the task execution time under this directory is also stated above, defined in/etc/crontab 6:25. I had a need before, look at the following configuration

/data/log/owan_web/chn_download_stat/chn_app_rec.log {

Copytruncate

# Weekly comments But will inherit/etc/logrorate.conf global variables, also weekly

Missingok

Rotate 10

Compress

Delaycompress

size=1000m # Size to get to sizes start dump

Notifempty

Create 664 Www-data Ymserver

Su Root

Dateext//This parameter is very important! Is the cut log file with the current date as the end of the format, such as xxx.log-20131216, if commented out, cut out by the number increment, that is, the xxx.log-1 of the previous format

Compress//Whether the post-dump log file, such as xxx.log-20131216.gz, is compressed by gzip, or if compression is not required, comment out the line

}

Then go to root crontab and configure a 0-point task.

[Email protected]:/etc/logrotate.d$ sudo crontab-l-u root

0 0 * * */USR/SBIN/LOGROTATE/ETC/LOGROTATE.D/WEB_ROTEATE-FV >/tmp/logro.log 2>&1

Because the logrotate cutting cycle is weekly, each cut is based on the time of the last cut, if the distance last one week, it will be cut, but we set the crontab daily cut, neither into the/etc/cron.daily/of the Daily Cut, And will not be cut weekly. This way, you can customize the cut log time you want perfectly.

Logrotate parameter Description

Compress post-dump log after gzip compression
Nocompress do not do gzip compression processing
Create Mode owner group rotation specifies the properties for creating a new file, as created 0777 nobody nobody
Nocreate do not create a new log file
When Delaycompress and compress are in use, the dump log file is compressed until the next dump
Nodelaycompress overrides the delaycompress option, the dump is compressed at the same time.
Missingok If the log is missing, continue scrolling the next log without error
Ifempty even if the log file is empty, this is the default option for Logrotate.
Notifempty The log file is empty, do not rotate
Mail address sends dump log files to the specified e-mail addresses
Olddir directory dumps the log file into the specified directory and must be in the same file system as the current log file
Noolddir dump log file and current log file in the same directory
Sharedscripts runs the Postrotate script, which executes the script uniformly once all the logs are rotated. If this is not configured, the script will be executed once per log rotation
Prerotate instructions that need to be executed before logrotate dumps, such as modifying the properties of a file;
Postrotate the instructions that need to be executed after the logrotate dump, such as restarting (KILL-HUP) a service! Must be independent
Daily Specify a dump cycle of daily
Weekly specify the dump cycle as weekly
Monthly specify a dump cycle of monthly
Rotate count Specifies the number of dumps before the log file was deleted, 0 means no backup, 5 means 5 backups reserved
Dateext using the current date as a naming format
DateFormat.%s with Dateext, immediately following the next line, define file name after file cut, must be used in conjunction with Dateext, only support%Y%m%d%s of four parameters
Size (or minsize) log-size the log file to the specified size before dumping, log-size can specify bytes (default) and KB (Sizek) or MB (sizem).
The log file is dumped when >= log-size. The following is a valid format: (Other format unit case not tried)
Size = 5 or size 5 (>= 5 bytes on dump)
Size = 100k or size 100k
Size = 100M or size 100M

One of the notable configurations is: Copytruncate

Copytruncate If you do not have this option, do it by moving the original log file into an old file like Log.1, and then creating a new file. If set, how to: Copy the original log file and turn it into a file of size 0.

The difference is if the process, such as Nginx used a file write log, no copytruncate words, cut the log, the old log log->log.1, and then create a new log. This time the Nginx Open file Description Fuijan log.1, by no signal to notify Nginx to change the log descriptor, so it will continue to write to log.1 log, so it does not meet our requirements. Because we want to cut the log, Nginx automatically writes the log to the new log file instead of the old Log.1 file

There are two solutions:

1. Configure the Nginx cutting log above and write a script inside the Postrotate

Postrotate # Instructions that need to be executed after the logrotate dump, such as restarting (KILL-HUP) a service! Must be independent

[-s/run/nginx.pid] && kill-usr1 ' cat/run/nginx.pid '

Endscript

This is to send a signal to Nginx, let Nginx close the old log file descriptor, reopen the new log file description, and write to the log

2. Using the Copytruncate parameter, to the above, after configuring it, the operation is to copy a log copies Become log.1, and then empty the contents of the log, so that the size of 0, then log is still the original old log, to the process (Nginx), still open is the original file descriptor, you can continue to write the log inside, without sending a signal to Nginx

Copytruncate This mode of operation, there is a time difference between copying and emptying, and some log data may be lost.

Nocopytruncate The backup log file is not truncated.

Read the original

Linux Log timing polling process in detail (logrotate)