Apche Log Series (5): Advanced Technology

This is the last article in the Apache Log Series. In addition to the previous articles, we also discussed three questions: How to Write log records to a specified program rather than a log file, how to rotate logs to prevent insufficient disk space and manage log files in Multi-virtual host environments.

1. Write log records to a specified program

In the previous article in this Apache Log Series, we discussed several Log File analysis tools. It should be noted that it does not list all analysis tools. Search for "Apache Log reporting" or similar keywords on Google and return hundreds of pages related to the topic, many vendors are selling their own solutions for this relatively simple problem.

Log records can not only be written to files, but also to specified processes. This is useful when we want to write log information into the database or some programs that can display website traffic statistics in real time.

So how can we achieve this? Using the transferlog or customlog command, we can specify "|", followed by the program name that receives the log information. For example:

Customlog |/usr/bin/apachelog. pl common

Here,/usr/bin/apachelog. pl is a program that knows how to process records of Apache log files. In fact, this program is very simple. For example, it can be a Perl program that processes log records in some way, or a program that writes log records to the database.

When using this method to record log data, security issues are the most important issues. The log file is opened by the user who starts the server, usually root. This is also effective for programs that write log records to the database. Therefore, it is necessary to ensure that the programs used to record log data have sufficient security.

If the log data is recorded through an insecure Program (this program may be intruded and modified by non-root users ), then the system is at risk of replacing the logging program with other malicious programs. For example, if/usr/bin/apachelog. PL can be modified by users all over the world, so any user can edit this file to close the web server, send the password file to a mailbox, or delete some important files, because the root user has the permission to perform all these operations.

If you want to write logs to a program, we recommend that you first check whether there are ready-made modules that have the functions you want. Visit http://modules.apache.org/. the website collects information about how to complete the tasks of different types of tasks to apache.

Ii. Rotating logs

The log file will become larger and larger. If you accidentally place the log file in a location such as/var, the log file may be full of partitions, resulting in the server being forced to stop running. This kind of thing has indeed happened.

To prevent this problem, move the log files to other locations with sufficient space before they become too large. This can be achieved through several methods. Some UNIX variants provide a logrotate script that can help us complete this task. For example, RedHat is pre-configured. It rotates log files every several days based on the log file size or the log file usage time.

To implement this function on our own, we can use the Perl module called logfile: Rotate (which can be downloaded from CPAN ). The following code has this function, which is run by cron at a certain interval (for example, a week) to save space. Each backup log file is compressed.

Use logfile: rotate;

$ Logfile = new logfile: Rotate (

File => & single;/usr/local/Apache/logs/access_log & single ;,

Count => 5,

Gzip => & single;/bin/gzip & single ;,

Signal => sub {

'/Usr/local/Apache/bin/apachectl restart ';

}

);

There are not many codes. The Perl module logfile: Rotate is responsible for all specific operation tasks. To run this program, we will get the files named access_log.1.gz and access_log.2.gz. It can help us avoid disk space shortage, so that we can save any number of file files.

3. logs of multiple virtual hosts

Several people once asked how to analyze logs when multiple virtual hosts are running on the same machine? I think they first save the logs of all virtual hosts to the same machine, and then try to split the log files into multiple parts according to the differences of the virtual hosts.

To completely solve this problem, do not write the logs of all virtual hosts to the same file at the beginning. Although I know that such tools do exist, they can separate the logs of multiple virtual hosts according to the virtual host configuration, identify which requests are sent to which virtual host, and then generate reports separately. However, this method seems too troublesome.

When you specify log files for each virtual host, you only need to specify the log files for the host in each virtualhost region. After that, when we need to prepare a report, we can process each log file separately.

However, you must pay attention to the issue of available file handles. That is to say, if there are hundreds of virtual hosts running on a server and each virtual host has a separate log file, the system may encounter insufficient file handles, it may cause system instability or even system crash. However, we need to pay attention to this issue only when the number of virtual hosts running on the server is very large.

The Apache Log Series have all ended. In these five articles, we have discussed all aspects of Apache Log functions, from standard logs (access logs and error logs) to custom logs and log analysis, I hope this content will be helpful to you.