How "Linux notes" uses the Logrotate tool to automatically slice and dice log files in scrolling

In many real-world projects, the application will continue to write logs, and if there are no log libraries in the program code that support automatic sharding, such as by filesize or date cutting, the log files will grow to the G level very quickly. Operating large files on a single machine is very inconvenient for subsequent follow-up logs.

This article describes how to use the Logrotate tool to slice logs outside the application.

1. What is Logrotate?
Logrotate is a log-cutting tool that comes with most Linux systems, and the shell terminal enters "man logrotate" to see its introduction (partially excerpt from the following):
Logrotate is designed to ease administration of systems, that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file is handled daily, weekly, monthly, or when it grows too large.
A more complete introduction can be seen in the shell terminal of its man document, here not to repeat.
In short, we know that it can be used to cut a log file that is still rolling, where "still scrolling" means that the current log file is still being appended to the application for continuous write operations.

2. Logrotate's application Scenario
If the file is static (that is, no application is currently aligned to write), split is a more commonly used static file cutting tool, and its usage profile is described here, which is skipped here.
Logrotate is often used to cut "dynamic" files that are still being written, which supports automatic segmentation of files by time interval or file size (automatic rotation). The specific usage is described below.

3. How to use Logrotate
According to man Logrotate's instructions, logrotate usage is simple:

logrotate [-DV] [-f|--force] [-s|--state file] Config_file+

The option (s) that appears in [] is optional, we only need to provide a copy of the configuration file, the following example to explain the configuration file format.

# sample Logrotate config filecompress # global configuration items, gzip compressed files after slicing # A profile can contain multiple independent sections# where the file path to be sliced + {XXX } constitutes a separate section, each section can be configured with a tangent branch for that type of file # Note: Global configuration items are used for each section, unless the behavior of global configuration items is overridden in the section configuration # The configuration of the section below indicates that the automatic segmentation of the log is triggered by time interval (weekly), and the history data is saved only after the last 5 # shards have been completed, and the daemon process is restarted by sending the-hup signal to restart the syslogd. /var/log/messages {Rotate 5 weekly postrotate/sbin/killall-hup syslogd Endscript} # The following example S Ection indicates that the log auto-segmentation is triggered by file size, in addition to the k shown above, the size unit can be either M or g# the file path to be sliced can have multiple, multiple paths separated by spaces "/var/log/httpd/access.log"/var/log/httpd/ Error.log {rotate 5 mail [email protected] size=100k sharedscripts # indicates that the HUP signal only needs to be sent once after all the files have been split, and if no such configuration is completed, each The Shard of a file executes a command configured between Postrotate/endscript once Postrotate/sbin/killall-hup httpd Endscript} # The following Example section table The time (monthly) Trigger Shard # Segmentation history file will be saved to Olddir specified directory (the directory needs to mkdir out beforehand, otherwise will error)/var/log/news/news.crit {monthly rotate 2 oldd Ir/var/log/news/old Missingok # If the file you specify at the beginning of the section does not exist, you will not get an error (default error) Postrotate Kill -hup ' Cat/var/run/inn.pid ' endscript nocompress # Splits without compression, overwriting the global configuration of the 1th line of the configuration file} 

In the above configuration file, "#" means that it is followed by a comment, which can have a single line, or it can be on the same line as the configuration item.
The configuration item "Rotate 5" indicates that the Shard history file holds the most recent 5 copies, and if this shard is triggered, there are 5 historical files (the default is messages.1/messages.2/messages.3/messages.4/ messages.5), the file messages.5 is physically deleted, and the remaining file suffix sequence is added 1.
The specific implementation of the segmentation can be referred to the "Bird's Linux private cuisine" here, the description.
The example configuration file above only shows the basic format of the Logrotate configuration file, which also supports a number of other configuration items, which can be referenced in the description of man logrotate.
The following configuration file describes how log auto-segmentation is done for applications that do not allow restart by file size in the actual project.

# filename:logrotate.conf/home/work/running.log {    copytruncate # Copy Running.log content to history file, then empty running.log content    Rotate    # Save 30 copies of the history log     size=1g      # If the log file reaches 1G, trigger the Shard    olddir/home/work/history # Specify the history log storage directory    Notifempty   # If/home/work/running.log is an empty file, do not slice    missingok    # If/home/work/running.log does not exist, do not error}

Special attention copytruncateThis configuration item, according to its documentation description (see excerpt below), It may result in partial data loss and should not be used in scenarios where log data loss is not allowed(If the log is not allowed to be lost, it is best to support the HUP signal in the application code to achieve graceful reboots, so you can avoid using copytruncate, and of course, you can use the Logrotate supported create configuration to achieve the goal.)
Copytruncate
Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally Creati Ng a new one, It can used when some program can is told to close their logfile and thus might continue writing (appending) to T He previous log file forever.Note that there was a very small time slice between copying the file and truncating it, so some logging data might be l OST.When this option is used, the Create option would have a no effect, as the old log file stays.

Resources
1. Man Logrotate
2. Bird's Linux private dish-the logrotate of a log file

====================== EOF ===================

How "Linux notes" uses the Logrotate tool to automatically slice and dice log files in scrolling