ApacheHTTP server usage -------- Log File

ApacheHTTP server usage -------- log file-Linux Enterprise Application-Linux server application information. The following is a detailed description. To effectively manage Web servers, it is necessary to report the activity, performance, and problems of the servers. The Apache HTTP Server provides comprehensive and flexible logging functions. This article describes how to configure a file and how to understand the log Content.
Top
Security Warning

Anyone who has the write permission to the directory where Apache stores log files can, of course, obtain the permissions of the user (usually root) who starts Apache, do not grant the write permission to anyone to store the log file directory. For details, see security tips.

In addition, log files may contain unconverted information from users, and users may have the opportunity to maliciously insert controllers. Therefore, you should be careful when processing raw logs.
Top
Error Log)
Related module commands


* ErrorLog
* LogLevel

An error log is the most important log file. Its file name and location depend on the ErrorLog command. Apache httpd stores diagnostic information and errors in processing requests in this file. Because the error details and solutions are often included, if the server is started or running, first, check the error log.

The error log is usually written into a file (generally error_log on unix systems and error. log on Windows and OS/2 ). In unix systems, error logs may also be redirected to syslog or transmitted to a program through pipeline operations.

The error log format is relatively flexible and can be appended with text descriptions. Some information may appear in the vast majority of records. A typical example is:

[Wed Oct 11 14:32:52 2000] [error] [client 127.0.0.1] client denied by server configuration:/export/home/live/ap/htdocs/test

The first is the date and time when the error occurred. The second is the severity of the error. The LogLevel Command records only errors higher than the specified severity level; the third option is the IP address that causes the error. Later, the information itself. In this example, the server rejects the access from this customer. When the server records the accessed files, it uses the file system path instead of the Web path.

The error log contains multiple types of information similar to the preceding example. In addition, any information output to stderr in the CGI script is recorded as debugging information in the error log.

You can add or delete error log items. However, for some special requests, there will also be corresponding records in the access log. For example, in the above example, there will also be corresponding records in the access log, whose status code is 403, because access logs can also be customized, you can get more information about error events from access logs.

It is useful to continuously monitor error logs for any issues during testing. In unix systems, you can do this:

Tail-f error_log
Top
Access Log)
Related module commands

* Mod_log_config
* Mod_setenvif



* CustomLog
* LogFormat
* SetEnvIf

Access logs record all requests processed by the server. The file name and location of the requests depend on the CustomLog command. The LogFormat command can simplify the log Content. This section describes how to configure Server access logs.

To implement log management, you must first generate access logs before analyzing logs to obtain useful statistics. Log analysis is not the responsibility of the Web server. It is beyond the scope of this article. For more information and information about analysis tools, see Open Directory or Yahoo.

Different versions of Apache httpd use different modules and commands to control access records, including mod_log_referer, mod_log_agent, and TransferLog commands. Now, the CustomLog command contains all the functions of the commands in the old version.

The access log format is highly flexible and uses a string in the C-style printf () function format. The following are several examples. The complete description shows the format string used for the mod_log_config module.
Common Log Format)

This is a typical record format:

LogFormat "% h % l % u % t \" % r \ "%> s % B" common
CustomLog logs/access_log common

It defines a specific record format string and gives it a common name. The "%" indicates that the server replaces the string with some information, and other characters are not replaced. Quotation marks (") must be escaped with a backslash to avoid the end of the string. The format string can also contain special control characters, such as line breaks "\ n" and tabs "\ t ".

The CustomLog command creates a new log file that uses the specified alias. Unless the file name is an absolute path starting with a slash, its path is relative to the ServerRoot path.

The preceding configuration is a common log format (CLF). It is used by many different Web servers and recognized by many log analysis programs, the records it generates are like:

127.0.0.1-frank [10/Oct/2000: 13: 55: 36-0700] "GET/apache_pb.gif HTTP/1.0" 200 2326

Each part of the record is described as follows:

127.0.0.1 (% h)
This is the IP address of the client sending the request to the server. If HostnameLookups is set to On, the server will try to resolve the Host Name of the IP address and replace the IP address here, but this is not recommended because it will significantly slow down the server, it is best to use a log processor to determine the host name, such as logresolve. If a proxy exists between the client and the server, the IP address in the record is the proxy IP address, not the real IP address of the client.
-(% L)
This is the RFC1413 identity determined by the client identd process. The output symbol "-" indicates that the information here is invalid. Unless in a strictly controlled internal network, this information is usually unreliable and should not be used. Apache will attempt to obtain this information only when the IdentityCheck command is set to On.
Frank (% u)
This is the customer ID (userid) that the HTTP authentication system obtains to access the Web page. The environment variable REMOTE_USER is set to this value and provided to the CGI script. If the status code is 401, it indicates that the customer has not passed the authentication, then this value is meaningless. If no password protection is set for the webpage, this option is "-".
[10/Oct/2000: 13: 55: 36-0700] (% t)
This is the time when the server completes request processing. The format is:

[Day/month/year: hour: minute: Second Time Zone]
Day = 2 digits
Month = 3 letters
Year = 4 digits
Time = 2 digits
Minute = 2 digits
Seconds = 2 digits
Time zone = (+ |-) 4 digits
You can use % {format} t in the format string to change the time output format. The format is the same as strftime () in the C standard library.
"GET/apache_pb.gif HTTP/1.0" (\ "% r \")
The quotation marks are the request lines sent by the client that contain many useful information. We can see that the customer's action is GET, the requested resource is/apache_pb.gif, and the protocol used is HTTP/1.0. You can also record other information, such as the format string "% m % U % q % H", which records the action, path, query string, and protocol, the output is the same as that of "% r.
200 (%> s)
This is the status code that the server returns to the client. This information is very valuable because it indicates the request result, or is successfully responded (starting with 2) or redirected (starting with 3 ), or an error occurs (starting with 4) or a server error (starting with 5 ). For a complete list of status codes, see HTTP specifications (Chapter RFC2616 10th ).
2326 (% B)
Finally, this is the number of bytes that are returned to the client, excluding the response header. If no information is returned, this option should be "-". If you want to record it as "0", % B should be used.

Combined Log Format)

Another common record format is the combined log format. The format is as follows:

LogFormat "% h % l % u % t \" % r \ "%> s % B \" % {Referer} I \ "\" % {User-agent} I \ "" combined
CustomLog log/access_log combined

This format is similar to the common log format, but it has two more % {header} I items. The header can be any request header. The format of the record is as follows:

127.0.0.1-frank [10/Oct/2000: 13: 55: 36-0700] "GET/apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I; Nav )"

Among them, the additional items are:

Http://www.example.com/start.html (\ "% {Referer} I \")
"Referer" request header. This indicates the webpage from which the request was submitted. The webpage should contain/apache_pb.gif or its connection.
"Mozilla/4.08 [en] (Win98; I; Nav)" (\ "% {User-agent} I \")
"User-Agent" request header. This is the browser identification information provided by the client.

Multi-file access logs

You can use multiple mmlog commands in the configuration file to create multi-file access logs. In the following example, the basic CLF information is recorded, and the information of the submitted webpage and browser is recorded. The last two lines of mmlog demonstrate how to simulate the ReferLog and AgentLog commands.

LogFormat "% h % l % u % t \" % r \ "%> s % B" common
CustomLog logs/access_log common
CustomLog logs/referer_log "% {Referer} I-> % U"
CustomLog logs/agent_log "% {User-agent} I"

This example also shows that the record format can be directly specified by CustomLog, rather than using LogFormat to create an alias.
Condition log

In many cases, it is convenient to selectively record some client requests based on the environment variables related to the request characteristics. First, you must use the SetEnvIf command to set specific environment variables to identify requests that meet certain conditions, and then use the env = clause of the CustomLog command, based on these environment variables, you can determine to record or exclude specific requests. For example:

# Do not record local requests
SetEnvIf Remote_Addr "127 \. 0 \. 0 \. 1" dontlog
# Do not log requests to the robots.txt File
SetEnvIf Request_URI "^/robots \. txt $" dontlog
# Record other requests
CustomLog logs/access_log common env =! Dontlog

Another example is to record requests in English to one log and non-English requests to another log:

SetEnvIf Accept-Language "en" english
CustomLog logs/english_log common env = english
CustomLog logs/non_english_log common env =! English

Although the above shows that conditional logging is powerful and flexible, it is not the only way to control log Content. You can also use the log processing program to remove content that you don't care about, this makes logs more useful.
Top
Log rolling

Even if a server is not busy, its log file contains a large amount of information. Generally, the access log is increased by 1 MB or more for every 10000 requests. In this case, it is necessary to regularly scroll log files. Apache keeps log files open and keeps writing information. Therefore, the server cannot scroll during running. After you move or delete a log file, you must restart the server to open the new log file.

An elegant (graceful) method can enable the server to enable new log files without losing the information that has not been written. Therefore, it is necessary to wait for a while for the server to complete the request being processed and write the record to the original log file. The following is a typical example of rolling logs and compressing old logs to save storage space:

Mv access_log access_log.old
Mv error_log error_log.old
Apachectl graceful
Sleep 600
Gzip access_log.old error_log.old

Another method for performing scrolling is to use the pipeline logs described in the next section.
Top
MPs queue logs

Apache httpd can pass access records and error information to another process through the pipeline, instead of writing a file. Because you do not need to program the master server, this function significantly enhances the flexibility of logs. You only need to use the pipeline operator "|" followed by an executable file name to obtain event records from the standard input device. Apache starts the pipeline log process at the same time, and if the process crashes during running, will restart this process (so we call this technology "reliable pipeline log ").

The pipeline log process is generated by its parent process Apache httpd and inherits its permissions. This means that pipeline processes are generally run as root, so it is critical to keep this program simple and secure.

An important use of pipeline logs is to allow log scrolling without restarting the server. To this end, the server provides a simple program rotatelogs. An example of rolling logs every 24 hours is as follows:

CustomLog "|/usr/local/apache/bin/rotatelogs/var/logs/access_log 86400" common

Note: The quotation marks are used to define the command line of the entire pipeline. Although this is for access logs, its usage is the same for other logs.

On other sites, there is a similar but more flexible log rolling program called cronolog.

If there is a simple offline log processing solution, condition logs and pipeline logs should not be used, even if they are very powerful.
Top
Virtual Host

If the server has several virtual hosts, there are several other functions to control log files. First, you can place the log command in The same access log and Error Log are used by the master server to record all requests and errors, but the information of each virtual host cannot be conveniently obtained.

If you place the CustomLog or ErrorLog command in All requests and error messages to this VM will be recorded in its private log file. The VM that uses the log command within the same segment will still use the same log as the master server. This method is very useful for servers with fewer virtual hosts. However, when there are many virtual hosts, it will lead to management difficulties and a shortage of file descriptors will often occur.

There is a good compromise between access logs. Access to All Hosts is recorded in the same access log file, and each record indicates the information of the virtual host, in the future, the records will be split and stored into different files. For example:

LogFormat "% v % l % u % t \" % r \ "%> s % B" comonvhost
CustomLog logs/access_log comonvhost

% V is used to append the VM information. There is a split-logfile program that can split logs based on different virtual host information and save the results to different files.
Top
Other log files
Related module commands

* Mod_logio
* Mod_log_forensic
* Mod_cgi
* Mod_rewrite



* LogFormat
* ForensicLog
* PidFile
* RewriteLog
* RewriteLogLevel
* ScriptLog
* ScriptLogBuffer
* ScriptLogLength

Records the actual number of bytes received and sent

Mod_logio adds two additional LogFormat fields (% I and % O) to record the actual number of bytes received and sent.
Comparison record (Forensic Logging)

Mod_log_forensic provides a comparison record of client requests, that is, two records are recorded before the request is processed and after the processing is completed, so the comparison log (forensic log) each request contains two records. The comparison recorder (forensic logger) is very strict and cannot be customized. It can be an invaluable debugging and security tool.
PID File

At startup, Apache httpd will save the process ID (process id [pid]) of its parent process httpd in the logs/httpd. PID file. The file name can be changed using the PidFile command. The PID can be used by the Administrator to restart or terminate the backend daemon. In Windows, you can use the command line parameter-k. For more information, see stop and restart.
Script log

To facilitate debugging, you can use the ScriptLog command to record the input and output of the CGI script. This function should be used only for testing and not for servers that work normally. For more information, see mod_cgi.
Re-write log

When using powerful and flexible mod_rewrite, it is almost necessary to use RewriteLog for debugging. This log provides detailed breakdown information about how the rewrite engine converts requests, depending on the RewriteLogLevel command.