httpd log and log rotation tool

This article directory:
1.1 Error log Errorlog
1.2 Access Log Customlog
1.3 Log Rotation: Rotatelogs tool
1.4 Log Rotation: Cronolog tool
Comparison of 1.5 rotatelogs and Cronolog

The log types that need to be logged are: Error log errorlog, access log customlog. Error log is generally used by default, up to the error log to the storage path, and Customlog because the volume is more, it is likely to need customization. In fact, the errorlog is provided by the HTTPD core module, while Customlog provides a specialized module mod_log_config to handle, the module also supports Transferlog instruction, which is basically similar to Customlog function and usage, If you need to check the official manual, this article skipped.

Finally, two kinds of log rotation tools are introduced: Apache httpd comes with rotatelogs tools and Cronolog tools.

1.1 Error log Errorlog

Error log levels Loglevel:debug, info, notice, warn (default), error, Crit, alert, Emerg.

Definition syntax for error logs:

ErrorLog file-path|syslog[:[facility][:tag]]

If you replace File-path with a syslog, it means that the Syslog log tool comes with the system, and facility is the type of facility that logs syslog logs. In general, this method is not used to log logs.

If you are using File-path, there are two ways to specify the file path directly, before using a pipeline, which means that the output log is passed as standard input to the post-pipeline log handler, such as the Rotatelogs tool that is passed to Apache. For example:

ErrorLog"logs/error_log""|/usr/local/apache/bin/rotatelogs /var/log/error_log 86400" common

Of course, for errorlog, the amount of data is not too much, generally directly using file records. For Customlog, you might consider using the log cutting tool for segmentation, rotation, and so on.

The record format of the error log is controlled by the Errorlogformat directive, for example, the following is the default record format for the error log in worker and event mode, where the meaning of each parameter is shown in the official manual Errorlogformat.

"[%{u}t] [%-m:%l] [pid %P:tid %T] %7F: %E: [client\ %a] %M% ,\ referer\ %{Referer}i"

1.2 Access Log Customlog

Use the Customlog directive to specify the location of the access log records, which can be used multiple times under the same host to represent the same log record to multiple locations. The syntax format is:

CustomLog file|pipeformat|nickname [env=[!]environment-variable| expr=expression]

Use Logformat to specify logging considerations, such as whether to log the client IP address, whether to log the request method, and so on. You can also use Logformat to define log classifications (known as nickname in httpd terminology), such as common classes, combined classes, and Combinedio classes. It supports very resilient recording matters, as detailed in the official manual Logformat.

Logformat"%h %l %u %t \"%r\ " %>s %b \"%{referer}< /c11>i\ "\"%{user-agent}i\ ""Combinedlogformat"%h %l %u %t \"%r\ " %>s %b"Common<ifmodule logio_module># need to enable MOD_LOGIO.C to use%I and%OLogformat"%h %l %u %t \"%r\ " %>s %b \" %{referer}i\ "\"%{user-agent}i\ " %I %O"combinedio</ifmodule>

Then use the Customlog directive to use the log classes that have already been defined. For example:

CustomLog"logs/access_log" combined

Of course, defining the classification using Logformat is just for convenience, and the Customlog directive simply defines the items to be logged without using nickname. For example:

"logs/access_log""%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""

If you supply the third parameter of Customlog, either env or expr, the log will be logged if the request satisfies the condition. For example, when a GIF file is requested, the log is logged independently, while other requests are recorded in another log.

SetEnvIf"gif-requests.log""nongif-requests.log" common env=!gif-image

In general, except for special needs, it is generally only possible to use the various items provided by Logformat. Otherwise, the logging system is a very disgusting and complex thing.

1.3 Log Rotation: Rotatelogs tool

This first describes the Rotatelogs tool, which is a simple tool that Apache HTTPD provides, based on time interval, size-splitting logs.

By default, it does not create the log file immediately when it is started, but only when a request arrives. The same is true for rotation, if the time interval of rotation is reached, and if no new requests arrive, a new log file is temporarily not created. If you use the "-f" option, the log file is created immediately at startup, regardless of whether or not there is a request. If you use the "-C" option, a new log file is created at each rotation, whether or not a new request arrives.

-l-f-e ] [ -c ] [ -n number-of-files ] logfile rotationtime|filesize(B|K|M|G)

Option Description:

• logfile: Specifies the logging path. Some modifiers can be added, such as the a.log_%y.%m.%d of the time class will append the suffix of the year, month, and day after A.log, and if no suffix decoration is given, automatically add ". nnnnnnnn", which means that the time point of creation is converted to a distance of 1970-01-01 The total number of seconds 00:00:00, which is "date +%s", is added to each rotation. If a relative path is used, it is relative to ServerRoot.
• -n: Use a number as a suffix, and a circular list of rotations, such as "-N 3", which is always recorded in the three files of Log.1, log.2, log.3, and no new log files are created.
• rotationtime: Specifies the time interval for rotation. Initializes the starting value to the corresponding time format. For example, when set to 3600, the rotation is indicated at the beginning of each hour, although the current point in time may only be 5 minutes into the next one hours, and a rotation will occur after 5 minutes.
• filesize: Turns the log in the size mode.
• -l: Calculates the time interval using local time. UTC time is used by default.
• -f: Forces the log file to open immediately. Because there are times when you start httpd, there may not be any requests arriving for a short period of time because no logs are required, so the log file is not created for the moment. Use this option to create a file immediately.
• -t: truncates the log instead of the rotation log. In this case, no more file suffixes will be added.
• -v: Verbose recording of information on rotation or truncation.
• -e: The log is also output to the standard error output. This option is useful when the log also needs to be processed by other tools.
• -c: A new file is created each time interval is reached, although no log arrives. By default, when the rotation interval arrives, if no log arrives, the log is temporarily not created, but waits until the first request arrives.

For more time class modifiers, see below.

Note that if the file name is duplicated at the time of rotation, such as rotation by 5M size, but the file name is formatted as "a.log_%y.%m.%d", if multiple rotations occur within the same day, the old file is overwritten because of the same file name.

In addition, Rotatelogs can only use the time class modifier for the log file name itself and cannot set it on the directory, otherwise it will fail to open the log when the httpd is started.

Example: The following uses the log pipe "|", which indicates that the logged log is passed to the following program (here, rotatelogs) for processing.

The $serverroot/logs/mylog.nnnnnnnn is replaced every day, where nnnnnnnn is the current time converted to a second value. And does not immediately create a log file at startup, nor does it create a new log file immediately after rotation.

CustomLog"|/usr/local/apache/bin/rotatelogs logs/mylog 86400" common

Follow the file size rotation.

CustomLog"|bin/rotatelogs /var/log/logfile 5M" common

Rotation error log, up to 5M when the rotation.

"|bin/rotatelogs /var/log/errorlog.%Y-%m-%d-%H_%M_%S 5M"

Truncate the log rather than the rotation log.

CustomLog"|bin/rotatelogs -t /var/log/logfile 86400" common

The following is the modifier for the date class. The following Cronolog tool also uses this same set of modifiers.

%% character n newline t horizontal tab Time class: H hours (xx.. at) I-Hour ( on.. AThe AM or PM ID under this locale is M-minute (xx.. -) s seconds (xx.. A,whichAllows forLeap seconds) X the locale under the time representation (e.g.:"15:12:47") Z time zone. If the time zone is not determined, then the meaningless date class: A weekday name under the locale (e.g.: Sun.  Sat) A full name of the working day under the locale (e.g.: Sunday. Satur-ay) b The month abbreviation under this locale (e.g.: Jan. DEC) B full name of the month under this locale (e.g.: January. December) c The date and time of the locale (e.g.:"Sun Dec 14:12:47 GMT 1996") d The number of days in the month ( on.. to) j Number of days in the current year (001..366) m number of months ( on.. A) U the number of weeks in the current year, beginning with Sunday as a week, where the first week begins with a Sunday week (xx.. -) W the number of weeks in the current year, with Monday as the beginning of the week, with the first week of the week including Sunday (xx.. -) W Working days (0..6,0Represents Sunday) x the date representation under the locale (e.g."13/04/97") y two-digit year (xx.. About) Y four-digit year (1970..2038)

1.4 Log Rotation: Cronolog tool

You can download it from the Epel source or install it from GitHub:https://github.com/fordmason/cronolog.

[Root@xuexi~]# cronolog-hUsage:cronolog [OPTIONS] logfile-spec-h NAME,--hardlink=name maintain a hardLinkFrom NAME to currentLog-S NAME,--symlink=name Maintain a symbolicLinkFrom NAME to currentLog-P NAME,--prev-symlink=name Maintain a symbolicLinkFrom NAME to PreviousLog-L NAME,--Link=name same as-s/--symlink-H,--helpPrintThis and thenExit-P PERIOD,--period=period set the rotation PERIOD explicitly-d delay,--delay=delay set the rotation PERIOD delay -O,--once-only create single outputLogFrom Template ( notRotated)-xFILE,--debug=fileWriteDebug Messages to FILE (orto standard errorifFILE is"-")-A,--american American date formats-e,--european European date formats (default) -s,--start- Time=time starting Time-Z TZ,-- Time-zone=tz UseTZ forTimezone-v,--versionPrintVersion number, thenExit

The use of this tool is very simple, but it has to be understood by its rotation principle. Take the following settings as an example:

"|/usr/local/sbin/cronolog logs/%Y/%m/%d/access.log" combined

Because the relative path is used, it is relative to ServerRoot. Assuming ServerRoot is/usr/local/apache, the log here will be created in/usr/local/apache/logs/%y/%m/%d/access.log, where%Y,%m, and%d respectively represent the year, month, and day. These modifiers consist of a file name or a directory that is called a template.

The rotation principle of cronolog is: According to the current log file template point in time, and the current time to compare, if a part of the template and the current point of time is different, you need to rotation, rotation will automatically create the missing directory, and calculate the next time the rotation of the point. On rotation, close the current log file, create a new log file based on the current point in time, and open a new log file. Note, however, that when the Cronolog tool is created or replaced, if no request arrives, the log file is not created immediately, which is the same as the default for the Rotatelogs tool.

It looks simple, it's really simple, it just needs to be compared with the time modifier and the current point of time. For example, the above configuration, if the current point of time is 2017-10-01, is created in the following order, note that it will only be created when the request arrives, and will not be created without a request.

• Period 1: Created when the first request arrives
  • The logs/2017 directory exists and is created if it does not exist.
  • The LOGS/2017/10 directory exists and is created if it does not exist.
  • The logs/2017/10/01 directory exists and is created if it does not exist.
  • Create the Logs/2017/10/01/access.log file and write the log.
• Period 2: Created when the first request arrives in the early morning of 2017-10-02
  • The logs/2017 directory already exists and is not created.
  • The LOGS/2017/10 directory already exists and is not created.
  • The logs/2017/10/02 directory does not exist and is created.
  • Create the Logs/2017/10/02/access.log file and write the log.

will continue to follow this rule continuously.

That is, Cronolog defaults to the time interval for the rotation of the smallest unit. For example, when the minimum unit is%d, the minimum unit for%w will be the weekly rotation, the minimum unit is%s, and the rotation will be in seconds, depending on the rotation of the day. However, it is still necessary to note that a new log file is created at the current point in time only when the request arrives. For example, if you create a file 10.log in seconds, 10th seconds, the log file is closed for 11 seconds, but a new log file is not created immediately, assuming a 15.log, rather than 11.log, is created at the 15th second of the incoming request.

In addition, Cronolog can use the "-P N UNITs" option to explicitly specify the rotation interval, valid units UNITs: Seconds,minutes,hours,days,weeks and months. For example, "-p 5 minutes" means rotation every 5 minutes. It should be noted, however, that the value of n must be a higher units (excluding the maximum common factor), such as a length of 60 minutes for an hour, and a "1,2,3,4,5,6,10,15,20,30 minutes" when specifying minutes units, but cannot be specified as " 7,9,11,12,13,14 "et minutes.

If you do not want to turn, there are two ways to do this: do not use the time class modifier, or use the "-o" option. The same file will be written all the time.

Cronolog also supports dynamic symbolic link functionality, such as the following configuration, so that each access to the/usr/local/apache/logs/access_log can access the latest log files.

"|/path/to/cronolog --symlink=/usr/local/apache/logs/access_log /usr/local/apache/logs/%Y/%m/access_log" combined

Finally, Cronolog does not support log truncation.

Comparison of 1.5 rotatelogs and Cronolog

Both of these tools have their advantages and disadvantages.

1. On the log file path: Rotatelogs cannot use the time class modifier as a directory and can only be used on log file names. and Cronolog can. such as Logs/%y%m/%d/access.log.
2. In the log control force: rotatelogs more control of the log, it can control the start, rotation when the log file is created immediately. Cronolog only creates a log file only if a new request arrives.
3. On the feature: Rotatelogs is richer, it supports log truncation, and supports cyclic rotation within a certain number. The Cronolog can only be wheeled, and can only be replaced by time.
4. On the convenience of log access: Cronolog through its dynamic symbolic link function makes this more convenient than the rotatelogs.

In fact, these are small problems.

back to Linux series article outline: http://www.cnblogs.com/f-ck-need-u/p/7048359.htmlBack to Site Architecture series article outline: http://www.cnblogs.com/f-ck-need-u/p/7576137.htmlback to Database series article outline: http://www.cnblogs.com/f-ck-need-u/p/7586194.htmlReprint Please specify Source: http://www.cnblogs.com/f-ck-need-u/p/7635409.htmlNote: If you think this article is not bad please click on the lower right corner of the recommendation, your support can inspire the author more enthusiasm for writing, thank you very much!

httpd log and log rotation tool