This completes cronolog configuration and installation. By default, cronolog is installed under/usr/local/sbin.
The command to modify apache Log configuration is as follows:
CustomLog "|/usr/local/sbin/cronolog /www/logs/secfocus/%w/access_log" combined
|
Here, % w indicates that logs are saved in different directories Based on the date and day of the week. In this way, logs of the week are saved. To perform log analysis, you need to copy (or move) the log file every day, if you do not want to save the log for one week, to a fixed location to facilitate processing of the log analysis statistics file, add a scheduled task as follows:
5 0 * * * /bin/mv /www/logs/secfocus/`date -v-1d +\%w`/access_log /www/logs/secfocus/access_log_yesterday
|
In this way, the log statistics and analysis program is used to process the file access_log_yesterday.
For large websites that use Server Load balancer technology, the access logs of multiple servers are merged. in this case, each server cannot use access_log_yesterday when defining or moving log files. Therefore, the server number should be included, such as the server IP address, to distinguish between them. Then run the website image and backup service rsyncd on each server (refer to "use rsync to implement website image and backup", ttp: // www.linuxaid.com.cn/engineer/ideal/article/rsync.htm ), then, download the daily installation configuration file of each server through rsync to the server dedicated for access statistics and analysis for merging.
Merge the log files of multiple servers. For example: log1 log2 log3 and output to log_all:
Sort-m-t ""-k 4-o log_all log1 log2 log3-m: sort by time using merge optimization algorithm-k 4, -o: stores the sorting result to the specified
File.
|
V. installation and configuration of the log statistics analysis program webalizer
Webalizer is an efficient and free web server log analysis program. The analysis result is in HTML file format, which allows you to conveniently browse through the web server. Many websites on the Internet use webalizer for web server log analysis. Webalizer has the following features:
1. It is a program written in C, so it has a high operating efficiency. On a machine with a clock speed of 10000 MHz, webalizer can analyze records per second. Therefore, it takes only 15 seconds to analyze a 40 m log file.
2. webalizer supports the standard Common Logfile Format. In addition, it also supports variants of several Combined log formats (Combined Logfile Format, this allows you to collect statistics on the customer's situation and the type of the customer's operating system. Now webalizer supports the wu-ftpd xferlog format and squid log file format.
3. Supports command line configuration and configuration files.
4. You can support multiple languages or perform Localization on your own.
5. Supports multiple platforms, such as UNIX, linux, NT, OS/2, and MacOS.
This is the first page of the access statistics analysis report generated by webalizer. It contains the table and bar chart statistics and analysis of the average access volume of each month. Click the minute of each month to obtain detailed statistics for each day of the month.
5.1 Installation
Before installation, make sure that the system has installed the gd library. You can use:
[root@mail root]# rpm -qa|grep gdgd-devel-1.8.4-4gdbm-devel-1.8.0-14gdbm-1.8.0-14sysklogd-1.4.1-8gd-1.8.4-4
|
To confirm that the system has installed two rpm packages, gd-deve and gd.
You can install webalizer either by downloading the source code or directly using the rpm package.
Installing the rpm package is very simple. Find the webalizer package from rpmfind.net and download it later:
rpm –ivh webalizer-2.01_10-1.i386.rpm
|
You can install it.
For the source code method, download the source code package from http://www.mrunix.net/webalizer/, and then install the package:
tar xvzf webalizer-2.01-10-src.tgz
|
There is an lang directory in the generated Directory, which stores various language files, but only the Traditional Chinese version, you can convert it into simplified, or re-translate it yourself. Then enter the generated directory:
cd webalizer-2.01-10./configuremake --with-language=Chinesemake install
|
After compilation, A webalizer executable file is installed in the/usr/local/bin/directory.
5.2 configure and run
To control webalizer running, you can use the configuration file or specify parameters in the command line. The configuration file is simple and flexible, and is suitable for the application environment of automatic web server log statistics and analysis.
The default configuration file of webalizer is/etc/webalizer. conf. When the "-f" option is not used When Webalizer is started, Webalizer searches for the file/etc/webalizer. conf, you can also use "-f" to specify the configuration file (when the server has a virtual host, you need to configure multiple different webalizer configuration files, the webalizer of different virtual hosts uses different configuration files. The configuration options to be modified in the Webalizer. conf configuration file are as follows:
LogFile /www/logs/secfocus/access_log
|
Indicates the path information of the configuration file. webalizer uses the log file as the input for statistical analysis;
OutputDir /www/htdocs/secfocus/usage
|
Used to indicate the directory for saving the generated Statistical Report. We used alias before to enable users to use http://www.secfocus.com/usage/to compile the statistical report.
HostName www.secfocus.com
|
Indicates the host name, which is referenced in the statistical report.
You do not need to modify other options. After the configuration file is modified, You need to periodically generate daily statistical analysis for webalizer.
Run as root: crontab-e enters the scheduled running task editing status and adds the following tasks:
5 0 * * * /usr/local/bin/webalizer –f /etc/secfocus.webalizer.conf15 0 * * * /usr/local/bin/webalizer –f /etc/tomorrowtel.webalizer.conf
|
Here we assume that the system runs two virtual hosts and define the Log Analysis configuration files secfocus. webalizer. conf and tomorrowtel. webalizer. conf respectively. In this way, we define statistical analysis of secfocus logs at a.m. and statistical analysis of tomorrowtel logs at a.m.
Then use the second day.
6. Protect log statistical analysis reports from unauthorized user access
We certainly do not want our Website access statistics to be browsed by others at will. Therefore, we need to protect the usage directory and only allow access by legal users. The basic authentication mechanism provided by apache can be used here. After configuration, you need to provide the password to connect to this address.
