Log analysis tool Awstats analysis results static

Objective:

Awstats is a Perl based WEB log analysis tool that develops quickly on SourceForge, and a full log analysis allows Awstats to display your following information:

Number of visits, unique visitors,
Access time and last visit,
User authentication, recently authenticated access,
Peak times per week (pages, clicks, hours and weeks of kilobytes),
Domain name/country host Visitor (page, CTR, byte, 269 domain name/country detection, GEOIP detection),
List of hosts, most recently visited and unresolved IP addresses
Most of the incoming and outgoing pages,
File type,
Website compression tables (mod_gzip or mod_deflate),
Operating system (number of pages per os, Ctr, Bytes, OS detected),
Using the browser,
Robot Access (detects 319 robots),
Worm attacks (5 worm families),
Search engine, use keyword search to find your address,
HTTP protocol error (recent lookup of pages not found),
Other personalized reports based on URLs, link parameters, relate to the purpose of the Integrated marketing field.
Your website is added "favorite Bookmark". Number of times.
Screen size (you need to add some HTML tags to the index page).
Browser support scale: Java, Flash, RealG2 Reader, Quicktime Reader, WMA Reader, PDF reader.
Load-balanced server ratio cluster report.

Awstats is running with the support of the PERL environment, and its support for Apache HTTP server is perfect from Awstats's documentation, and when we replace the WEB server with Nginx, it becomes cumbersome to run Awstats. First of all, Nginx's own support for Perl is weak, even officially not recommended, and the log format needs to be modified before it can be run.
Using Awstats, you can analyze Apache logs, as well as analyze Nginx logs. This article will introduce the visit log of automatic timing cutting nginx, and use Awstats to analyze Nginx log and realize statistic result to be safe and convenient to inspect.

Environment:

CentOS 6.4 x86_64
ip:192.168.1.113
Domain Name: www.sunsky.com (server and client are parsed through Hosts file)
nginx-1.2.9 compile installation, path/usr/local/nginx, service open state
The logging format is nginx default, do not change, or cause Awstats to parse the log.
Log_format Main ' $remote _addr-$remote _user [$time _local] ' $request '
' $status $body _bytes_sent ' $http _referer '
' $http _user_agent ', ' $http _x_forwarded_for ';
Awstats-7.2.tar.gz

One, automatic log cutting

For nginx log cutting, because there is no like Apache to use the Cronolog tool, here we write a script, so that it can be automatically executed at 00:01 every day, cutting yesterday's log (to Awstats analysis), compress the log of the day before yesterday (compressed log can reduce storage space, To prevent Awstats from being compressed, it only compresses the logs from the day before yesterday.

vim/server/scripts/cut_nginx_log.sh

Enter the following content:

#!/bin/sh
Yesterday= ' date-d "Yesterday" + "%y%m%d"
Before_yesterday= ' date-d '-2 day ' + '%y%m%d '
Nginx_dir= "/usr/local/nginx"
Nginx_logs= "/app/logs"
Log_name= "Www_access"
Cd/tmp
[-D $Nginx _logs] && cd $Nginx _logs | | Exit 1
[f $Log _name.log] &&/bin/mv $Log _name.log ${log_name}_${yesterday}.log | | Exit 1
If [$?-eq 0-a-F $Nginx _dir/logs/nginx.pid]
Then
KILL-USR1 ' Cat $Nginx _dir/logs/nginx.pid '
Fi
[f ${log_name}_${before_yesterday}.log] &&/usr/bin/gzip ${log_name}_${before_yesterday}.log| | Exit 1

Execute crontab-e to add the script to the scheduled task

1 0 * * */bin/sh/server/scripts/cut_nginx_log.sh >/dev/null 2>&1

So that every day 00:01 can automatically realize the log cutting, compression and other functions

Because this experiment under the nginx at this time already has the log, in addition in order to post Awstats can analyze the log which has been cut, so here we want to run this script, to cut the existing log to produce yesterday's log convenient after the text operation.

/bin/sh/server/scripts/cut_nginx_log.sh >/dev/null 2>&1

Ii. installation and configuration of Awstats

1. Deployment of Awstats
First we download the Awstats package and place it under the regular directory (/usr/local)

wget http://awstats.sourceforge.net/files/awstats-7.2.tar.gz
Tar zxf awstats-7.2.tar.gz
MV Awstats-7.2/usr/local/awstats

Because the permissions in the package downloaded by wget are not root, you have to modify the permissions here, otherwise *.pl will not be able to run later

Chown-r Root.root/usr/local/awstats
chmod +x/usr/local/awstats/tools/*.pl
chmod +x/usr/local/awstats/wwwroot/cgi-bin/*.pl

Next we are going to execute the awstats_configure.pl Configuration Wizard under Awstats/tools to generate Awstats configuration files, awstats the naming convention for the profile is awstats.website.conf.

cd/usr/local/awstats/tools/
./awstats_configure.pl

The following prompts appear

-----AWStats Awstats_configure 1.0 (Build 1.9) (c) Laurent destailleur-----
This tool would help you to the Configure AWStats to analyze statistics for
One Web server. You can try to use it for let it does all this is possible
In AWStats Setup, however following the "step" by step Manual setup
Documentation (docs/index.html) is often a better idea. Above All if:
-You are are not a administrator user,
-Want to analyze downloaded log files without Web server,
-Want to analyze mail or FTP log files instead of Web log files,
-Need to analyze load balanced servers log files,
-Want to ' understand ' all possible ways to use AWStats ...
Read the AWStats documentation (docs/index.html).
-----> Running OS detected:linux, BSD or Unix
-----> Check for Web Server install
Enter full config file path of your WEB server.
Example:/etc/httpd/httpd.conf
Example:/usr/local/apache2/conf/httpd.conf
Example:c:\program Files\apache group\apache\conf\httpd.conf
Config file path (' None ' to skip Web server Setup):
> None #这里让填写网页服务器的配置文件路径 because we don't use Apache, so we're going to fill in none here.
Your Web server config file (s) could not is found.
You'll need to the setup your Web server manually to declare AWStats
Script as a CGI, if you are want to build reports dynamically.
AWStats Setup documentation (file docs/index.html)
-----> Update Model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf '
File awstats.model.conf updated.
-----> Need To create a new config file?
Do and want me to build a new AWStats Config/profile
File (required if install) [y/n]? Y #询问是否创建一个新的配置文件, fill in y here.
-----> Define Config file name to create
What is the name of your Web site or profile?
Example:www.mysite.com
Example:demo
Your Web site, virtual server or profile name:
> www.sunsky.com #这里让填写你的网站域名, Virtual host name or random configuration name
-----> Define Config file path
In which directory your config file (s)?
Default:/etc/awstats
Directory path to store config file (s) (Enter for default):
> #这里要填写你配置文件存放路径, we use its default path/etc/awstats, so direct return can
-----> Create config file '/etc/awstats/awstats.www.sunsky.com.conf '
Config file/etc/awstats/awstats.www.sunsky.com.conf created.
-----> ADD Update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
Can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl-update-config=www.sunsky.com
Or If you have several config files and prefer have only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue ... #提示不能自动加入crontab定时任务, you need to add it later, we can continue by pressing ENTER
A Simple config file has been created:/etc/awstats/awstats.www.sunsky.com.conf
You are should have a look inside to check and change manually main parameters.
can then manually update your statistics for ' www.sunsky.com ' with command:
> Perl awstats.pl-update-config=www.sunsky.com
can also build static on pages for ' www.sunsky.com ' with command:
> Perl awstats.pl-output=pagetype-config=www.sunsky.com
Press ENTER to finish ... #提示配置文件创建完成和如何更新配置及建立静态报告页, here we can end this configuration wizard