Ubutun under Webalizer analysis Apache log

http://www.webalizer.org/ configuration Webalizer

We can configure Webalizer from the command line, or through a configuration file. The following highlights the use of configuration files for configuration, which is more intuitive to use, and generally used.

The path to the Webalizer configuration file is "/etc/webalizer/webalizer.conf.sample. In general, the configuration file of the default parameter configuration can meet a certain application needs, can be used directly. Next, we'll show you how to configure the file as an instance.

Edit the Webalizer configuration file using VI.

Webalizer/webalizer.conf

Sets the storage path for the access log.

Set the system default value:

Change to the following content:

Set the report output directory.   from default: use the crontab command to let Webalizer generate daily traffic statistics for Apache every day.   Add the following configuration line, which causes Webalizer to generate the Squid traffic statistics for the day every 9 o'clock in the evening. View page as follows: 0 * * */usr/bin/webalizer-c/etc/webalizer/webalizer.conf

Network traffic log analysis is important for network administrators. Through the system record of traffic log, the administrator can clearly see the user's use of the network server, can dig and discover the network security problems.

There are many log and traffic analysis software under Linux. Webalizer is one of the most efficient, easy-to-use, free Web server logs and traffic analysis programs. Many sites on the Internet use it for Web server logs and traffic analysis. This article will be based on the Linux agent Server Squid traffic log analysis as the background, detailed how to use the software for traffic analysis.

Webalizer has some of the following features:

Written in C language, it has high running efficiency. On a 200Mhz machine, the Webalizer can analyze 10,000 records per second, so it takes 15 seconds to parse a 40M-sized log file.

Supports Standard General log file format (Common Logfile format); In addition to this, several variants of the combined log format (Combined Logfile format) are supported to account for customer situations and customer operating system types. And now Webalizer can support WU-FTPD xferlog log format as well as squid log file format.

Installation Configuration Webalizer

1. Installation of the source code package

Download Webalizer from Webalizer's site http://www.mrunix.net/webalizer/download.html, the latest version is webalizer-2.01-10-src.tgz. Follow these steps to install:

Unzip the open source code package

#tar Xvzf webalizer-2.01-10-src.tgz

Then go to the generated directory

#./configure

#make--with-language=chinese

After the compilation is successful, a webalizer executable is generated that can be copied to the/usr/sbin/directory

#cp webalizer/usr/sbin/

In this way, the installation succeeds and you can configure it.

2. Installation of RPM packages

Webalizer's RPM installation package is available in Red Hat Linux 9, and the steps to install the RPM package are described below:

#rpm-qa | grep Webalizer//See if the Webalizer is already installed

#mount/mnt/cdrom//Insert the first installation disc into the optical drive and mount the disc partition

#cd/mnt/cdrom/red Hat/rpms

Enter the/mnt/cdrom/red Hat/rpms directory

#rpm-IVH webalizer-2.01_10-11.i386.rpm

Performing the installation

The user can configure the Webalizer through the command line or through the configuration file, the following describes the use of configuration files for configuration, the method is more intuitive use of the form is more common.

The path to the Webalizer configuration file is:/etc/webalizer.conf, which has a corresponding example file/etc/webalizer.conf.sample. In general, the default options for this profile can be used directly for certain applications. The actual examples and steps for configuring the file are given below:

1. Editing Webalizer profiles with VI

#vi/etc/webalizer.squid.conf

2. Set the access log's storage path and change the system default: Logfile/var/log/httpd/access_log to the following:

Logfile/var/log/squid/access.log

3. To set the format type of the access log, modify the LogType CLF to:

LogType Squid

4. Save the directory by setting the report output directory from the default value of Outputdir/var/www/html/usages to a statistics report for squid. Use the crontab command to allow Webalizer to generate squid daily traffic statistics analysis.

#crontab-E

5. Add the following configuration line, so that webalizer every 9 o'clock in the evening to generate the day's Squid traffic statistics analysis results:

0 * * */usr/bin/webalizer-c/etc/webalizer.conf

Squid Log Management Introduction

Squid is Linux's next proxy server for caching Internet data, which receives a user's download request and automatically processes the downloaded data. In the network environment, the local network administrator can often delegate access to many resources of the Internet, so it is of great significance to log audit and management.

Squid Log management features more powerful. By viewing the relevant log files, the user can effectively manage the proxy server by providing real-time and accurate information about the access records of the proxy server, the cache's storage status, and the cache's access history.

Squid The most important log file is the access log, the access path is:/var/log/squid/access.log. This log file records many useful information about a client's use of a proxy server, including usage time, requested content, client information, and so on. The format of the file is as follows:

Time eclapsed remotehost Code/status bytes Method URL rfc931 peerstatus/peerhost type

As you can see, the log file records contain a total of 10 fields, each of which has the following meanings:

Time: The number of seconds that the client Access Proxy was logged, from January 1, 1970 to the date of the visit, to milliseconds.

Eclapsed: The amount of time it takes to log processing of the cache, counted in milliseconds.

RemoteHost: Logs the IP address or domain name of the access client.

Code/status: Result information encoding/status information encoding, such as tcp_miss/205.

Bytes: number of bytes cached.

Method:http request method, get or post.

URL: The URL of the destination address to access, such as www.sina.com.cn.

Rfc931: Default, not currently used.

Peerstatus/peerhost: Cache level/Destination IP, such as direct/211.163.21.19.

Type: Cache object type, such as text/html.

The following shows a real-world access log in the actual application, with only a subset of them removed:

#cd/var/log/squid

#cat Access.log

1119263842.955 195 192.168.10.3 tcp_miss/200 1286 GET http://ad4.sina.com. Cn/sina/ae/ad_src/popup/pops1.html? -Direct/218.30.108.19 text/html

1119263843.241 286 192.168.10.3 tcp_miss/200 44689 GET http://ad4.sina.com. cn/200506/15/21135_300x250. gif-direct/218.30.108.18 Image/gif

Take the first log record as an example, with the meanings of each field as follows:

The Client Access Proxy time is 1119263842.955 milliseconds. The time it takes to process the cache is 195 milliseconds. The IP address of the client is 192.168.10.3, which is the intranet address. The result information encoding/status information is encoded as tcp_miss/200. The number of cache bytes is 1286. The HTTP request method is get. The URL of the destination address for the visit is: http://ad4.sina.com.cn/sina/ae/ad_src/popup/pops1.html. The cache level/destination IP is: direct/218.30.108.19. The Cache object type is text/html. rfc913 is not used, denoted by "-".

In the above 10 fields, information encoding and status information encoding is more important, which indicates that the proxy server's return result of the information requested by the client is directly related to the cache hit rate of the proxy server and the CAHCE process. This information can really reflect the overall performance of the proxy server cache, and has a large reference value.

View Squid network traffic log

Webalizer can be a large number of squid log information to synthesize statistics, to obtain a visual display results for network administrators or network users to reference and evaluate the use of Squid server, for making corresponding decisions have a considerable reference value.

In the use of the process should be noted: Because the use of the software is displayed through the Web page, so the connection with the Web server is relatively close. The following is an example of how this software is used in conjunction with squid servers. Typically, after you have configured the software, start the Web server (only start the server to view the details of the log statistics from the Web page) and the software:

#service httpd Start

#/usr/bin/webalizer-c/etc/webalizer.conf

After launching the Apache service and the software, you can view the traffic analysis results of Squid Proxy server through the browser, and enter the address of the Apache server in the address bar of IE browser: http://210.77.27.59/usage, the result is obtained.

Figure View Traffic statistics

The figure shows the flow analysis and some data description, more detailed statistical analysis data, you can click on the graph "Summary by Month" The "Jul 2005" hyperlink, then display more detailed information, by analyzing and viewing the results of these statistical charts, Users can clearly know the details of the usage of Squid Proxy server, and can make appropriate adjustments and optimizations to the server according to the situation.

Finally, there is one more thing to note about using Webalizer: There is usually no need for each user to have permission to view the traffic condition of the proxy server, otherwise it is unsafe for the system. Illegal users in order to conceal their illegal use of Proxy server facts, the results of traffic statistics can be modified, deleted and other operations. Therefore, we need to use an access control policy to control the view permissions of the above analytic chart, which can be configured for authentication and authorization in Apache's Master profile/etc/htpd/conf/httpd.conf, as follows:

< Directory "/var/www/html/usage" >

Use Basic authentication Methods

AuthType Basic

Specify the authentication domain name

AuthName "Admin"

Specify the location where the authentication password file is stored

Authuserfile/var/www/passwd/admin

Authorization to all users in the authentication password file

Require Valid-user

</directory>

After the authentication and authorization configuration is successful, you need to create the authentication password file and add the user.

#mkdir/var/www/passwd

#cd/var/www/passwd

#htpasswd-C Admin Liyang

New Password:

Re-type New Password:

Adding passwor for user Liyang

Then, change the owner of the password file to Apache and restart Apache:

#chown Apache.apache Admin

#service httpd Restart

After the above configuration, the user enters the path in the IE Address bar: http://210.77.27.59/usage, the user with authentication and authorization can view the traffic analysis report information of Squid Proxy server through the user authentication interface.

Installation II---------------------------------------------

Webalizer configuration is not very difficult, but for the novice, installation is very troublesome, this article will detail the installation process, help you step by step towards success!

Webalizer It takes only a second to process 10,000 records on a normal Pentium machine at a high Mhz, but not so fast on my Celeron machine. Probably I did the reverse resolution of the IP address for the sake of. However, regardless of speed, I think webalizer, although not comparable with some commercial software interface, but compared with other open source software, its statistical ability is much stronger.

One. Installation

Webalizer need to use the relevant JPEG, PNG and zlib libraries, which are generally not installed by default, so beginners will find it difficult to start. Because these libraries are relevant, the following installation steps cannot be reversed:

1, the installation of JPEG library
Fromhttp://www.ijg.org/After downloading the JPEG library, unpack the zip file and go to this directory:
./configure
Make
Make install
Cd..
2, the installation of Zlib library
Fromhttp://www.info-zip.org/pub/infozip/zlib/After downloading the zlib library,

Gunzip zlib*.tar.gz
TAR-XVPF Zlib-1.1.3.tar
MV zlib-1.1.3 zlib
CD zlib
./configure
Make
Make Test
Make install
Cd..

3, the installation of Libpng library


Fromhttp://www.libpng.org/After downloading libpng:

Gunzip libpng*.tar.gz
TAR-XVPF Libpng-1.0.10.tar
MV libpng-1.0.10 libpng
CD libpng
CP Scripts/makefile.linux Makefile
Make
Make Test
Make install
Cd..

You can also get thehttp://www.freetype.org/download.htmlChoose to download the FreeType font, but this step is not required.
CD freetype*
Make Setup
Make
Make install
Cd..

Here we can start installing the GD library:
Fromhttp://www.boutell.com/gd/G Download d-1.8.4.tar.gz:

CD gd-1.8.4

#如果需要使用 Xpm or TrueType support, modify Makefile before make.

Make

#很多人会经历找不到 Gd.h, it is recommended that you search the hard drive for the old gd.h version and make the install

Next, we can start installing Webalizer, fromhttp://www.mrunix.net/webalizer/After download:

CD webalizer-2.01-06
./configure
Make
Make install

If you do not install Webalizer by using source code compilation, you need to back up the/etc/webalizer.conf.

We also need to install Berkeley DB if we need to use the DNS reverse resolution feature. The specific installation can refer to the documentation: Dns.readme.
Two. Configuration:

A common configuration file is:
Hidereferrer www.freelamp.com
Incremental Yes
PageType htm*
PageType CGI
PageType PHP
PageType Pl
Ignorehist No
Countrygraph Yes
Indexalias index.cgi
Indexalias index.php
Hidereferrer Direct Request
Hideurl *.gif
Hideurl *. Gif
Hideurl *.jpg
Hideurl *. Jpg
Hideurl *.png
Hideurl *. Png
groupurl/cgi-bin/* CGI Scripts
groupurl/images/* Images
Groupreferrer yahoo.com/yahoo!
Groupreferrer Excite.com/excite
Groupreferrer Infoseek.com/infoseek
Groupreferrer Webcrawler.com/webcrawler
Groupreferrer Google.com/google
Groupreferrer Lycos.com/lycos
Groupreferrer Metacrawler.com/metacrawler
Groupagent MSIE Internet Exploder
Groupagent Mozilla Netscape
groupshading Yes
Mangleagents 3


Then we run/usr/local/bin/webalizer-q-c/root/webalizer.conf-n www.freelamp.com-o/home/freelamp/public_html/log//Home /freelamp/logs/freelamp.log writes this sentence to crontab, we canhttp://www.freelamp.com/log/See the log statistics results.

Install III--------------------------------------------

recently in the Linux Resin server log analysis, the online to webalizer this tool evaluation is very high, and it is free, so I installed.
    Requirements:
     Since webalizer is required to use the drawing when making the report, it is necessary to install the Libpng library before installing Webalzer, and the GD library from the Span style= "color: #000000;" > http://aleron.dl.sourceforge.net/sourceforge/libpng/ libpng-1.2.7-config.tar.gz   download gd-1.2.7.tar.gz test into a directory to extract
Tar-xfzv libpng-1.2.7.tar.gz
cd  libpng the directory where you extracted
./configure
Make
make install
------------- -libpng installation is complete---
now starts the GD library
from http://www.boutell.com/gd/http/gd-2.0.28.tar.gz   Download gd.2.0.28.tar.gz
Tar-xfzv gd-2.0.28.tar . GZ
CD GD extract directory
./configure
Make
make install
----------------GD Library installs-----
now installs Webalizer

1. Download Webalizer from Webalizer's official site http://www.mrunix.net/webalizer/ , the current version is webalizer-2.01-10-src.tgz.

2. First unlock the source code package:

Tar xvzf webalizer-2.01-10-src.tgz

3. There is a lang directory in the generated directory, which holds a variety of language files, which are available in traditional Chinese and Simplified Chinese versions.

4. Then go to the generated directory:

./configure
./configure Make--with-language=simplified_chinese
Make
Make install

5. After the compilation is successful, a webalizer executable file is generated that can be copied to the/usr/sbin/directory:

CP webalizer/usr/sbin/

You can then start configuring the Webalizer. (Configuration according to their actual situation, there are detailed comments)
Config file is/etc/webalizer.conf by default
Test:
Log in with Root, execute/etc/rc.d/webalizer
Normal prompt:

Using the log file/home/smp/resin/logs/access.log (CLF)
Output results to/home/smp/resin/webapps/itemlog
The host name that generated the report is ' 192.168.1.102:8080 '
Reading history information File ... webalizer.hist
Generating report----for November 2004
Generating consolidated reports
Saving history information File ...
287 Records 0.04 seconds

In order for Webzlizer to run as planned, all now I write a script: (in/etc/rc.c/webalizer)
Export home=/

Export Path=/usr/bin

Export Ld_library_path=/usr/local/lib
Export Nls_lang=american_america. Zhs16gbk
Export Classpath=/usr/local/lib

Export lang=zh_cn.gb2312i

#!/bin/sh
/usr/bin/webalizer >/etc/rc.d/out.log

Pay attention to the environment variables must match, otherwise cannot execute
Add in Crontab-e
* * * */etc/rc.d/webalizer-c/etc/webalizer.conf
Now it's done.

Ubutun under Webalizer analysis Apache log