Dynamic analysis Results of log analysis tool Awstats

Previous Blog "Analysis tools awstats actual combat nginx-analysis results of static" describes how to awstats log analysis information with static page for display, but the display effect is definitely not dynamic. This blog post will bring you together to deploy the dynamic analysis results lookup.

Environment:

CentOS 6.4
ip:192.168.1.113
Domain Name: www.sunsky.com (server and client are parsed through Hosts file)
nginx-1.2.9 compile installation, path/usr/local/nginx, service open state
The logging format is nginx default, do not change, or cause Awstats to parse the log.
Log_format Main ' $remote _addr-$remote _user [$time _local] ' $request '
' $status $body _bytes_sent ' $http _referer '
' $http _user_agent ', ' $http _x_forwarded_for ';
awstats-7.2.tar.gz cpan-2.00.tar.gz fcgi-0.74.tar.gz fcgi-procmanager-0.24.tar.gz
You must have perl-devel, or you cannot compile fcgi.

One, automatic log cutting

For nginx log cutting, because there is no like Apache to use the Cronolog tool, here we write a script, so that it can be automatically executed at 00:01 every day, cutting yesterday's log (to Awstats analysis), compress the log of the day before yesterday (compressed log can reduce storage space, To prevent Awstats from being compressed, it only compresses the logs from the day before yesterday.

vim/server/scripts/cut_nginx_log.sh

Enter the following content:

#!/bin/sh
Yesterday= ' date-d "Yesterday" + "%y%m%d"
Before_yesterday= ' date-d '-2 day ' + '%y%m%d '
Nginx_dir= "/usr/local/nginx"
Nginx_logs= "/app/logs"
Log_name= "Www_access"
Cd/tmp
[-D $Nginx _logs] && cd $Nginx _logs | | Exit 1
[f $Log _name.log] &&/bin/mv $Log _name.log ${log_name}_${yesterday}.log | | Exit 1
If [$?-eq 0-a-F $Nginx _dir/logs/nginx.pid]
Then
KILL-USR1 ' Cat $Nginx _dir/logs/nginx.pid '
Fi
[f ${log_name}_${before_yesterday}.log] &&/usr/bin/gzip ${log_name}_${before_yesterday}.log| | Exit 1

Execute crontab-e to add the script to the scheduled task

1 0 * * */bin/sh/server/scripts/cut_nginx_log.sh >/dev/null 2>&1

So every day 00:01 can automatically realize the log cutting, compression and other functions.

Because this experiment under the nginx at this time already has the log, in addition in order to post Awstats can analyze the log which has been cut, so here we want to run this script, to cut the existing log to produce yesterday's log convenient after the text operation.

/bin/sh/server/scripts/cut_nginx_log.sh >/dev/null 2>&1

Second, configure fcgi
1. Installation Cpan

wget http://search.cpan.org/CPAN/authors/id/A/AN/ANDK/CPAN-2.00.tar.gz
Tar zxf cpan-2.00.tar.gz
CD CPAN-2.00
Perl makefile.pl
Make && make install

2, Installation fcgi and fcgi::P Rocmanager

wget http://search.cpan.org/CPAN/authors/id/F/FL/FLORA/FCGI-0.74.tar.gz
Tar zxf fcgi-0.74.tar.gz
CD FCGI-0.74
The first method of installation: Perl-mcpan-e ' Install fcgi '
The second method of installation: Perl makefile.pl
Make&&make Install
wget http://search.cpan.org/CPAN/authors/id/B/BO/BOBTFISH/FCGI-ProcManager-0.24.tar.gz
Tar zxf fcgi-procmanager-0.24.tar.gz
CD fcgi-procmanager-0.24
The first method of installation: Perl-mcpan-e ' install fcgi::P rocmanager '
The second method of installation: Perl makefile.pl
Make&&make Install

When you perform the first installation method, you must automatically scroll down to prompt for OK. If you are prompted to enter Yes and so on, you need to follow the prompts after the operation, and then run the second time until the full automatic rolling down prompts OK to complete the installation. Or you can use the second method to perform the installation.

3. Create fcgi startup files

vi/usr/local/nginx/sbin/fcgi #此处按个人习惯命名 #!/usr/bin/perl use fcgi;
#perl-mcpan-e ' Install fcgi ' use Socket;
Use POSIX QW (SETSID);
#use Fcntl;
Require ' syscall.ph ';
&daemonize; #this keeps the program alive or something after exec's ing perl scripts End () {} BEGIN () {} *core::global::exit = sub {d ie "fakeexit\nrc=". Shift (). " \ n ";
};
Eval Q{exit};
if ($@) {exit unless $@ =~/^fakeexit/;};
&main;
    Sub Daemonize () {chdir '/' or die ' Can ' t chdir to/: $!;
    Defined (my $pid = fork) or die "Can ' t fork: $!";
    Exit if $pid;
    Setsid or Die "Can ' t start a new session: $!";
Umask 0;
        Sub Main {# $socket = Fcgi::opensocket ("127.0.0.1:8999", 10);
$socket = Fcgi::opensocket ("/usr/local/nginx/fastcgi_temp/perl_cgi-dispatch.sock", 10);
        #use UNIX Sockets-user Running this script must have W access to the ' Nginx ' folder!! $request = Fcgi::request (\*stdin, \*stdout, \*stderr, \%req_params, $SOCKET);
            if ($request) {request_loop ()};
Fcgi::closesocket ($socket);
                                                                                                                                                                              Sub Request_loop {while ($request->accept () >= 0) { #processing any STDIN input to WebServer (for cgi-post actions) $stdin _passth
           Rough = '; $req _len = 0 + $req _params{' ContenT_length '};
                if (($req _params{' request_method '} eq ' POST ') && ($req _len!= 0)) {my $bytes _read = 0;
                        while ($bytes _read < $req _len) {my $data = ';
                        My $bytes = Read (STDIN, $data, ($req _len-$bytes _read));
                        Last if ($bytes = = 0 | |!defined ($bytes));
                        $stdin _passthrough. = $data;
                $bytes _read + + $bytes; #running the CGI app if ((x $req _params{script_filename}) && #can I exec
                 Ute this?
                 (S $req _params{script_filename}) && #Is This file empty?
            (R $req _params{script_filename}) #can I read this file?
                ) {pipe (CHILD_RD, PARENT_WR);
                My $pid = open (Kid_to_read, "-|");
Unless (defined ($pid)) {print ("content-type:text/plain\r\n\r\n");                        Print "error:cgi app returned no output-executing $req _params {script_filename} failed!\n";
                Next
                        if ($pid > 0) {close (CHILD_RD);
                        Print PARENT_WR $stdin _passthrough;
                        Close (PARENT_WR);
                        while (my $s = <KID_TO_READ>) {print $s;}
                        Close Kid_to_read;
                Waitpid ($pid, 0); else {foreach $key (keys%req_params) {$ENV {$key} = $req _params{$ke
                        Y};  # CD to the script ' s local directory if ($req _params{script_filename} =~
                        /^ (. *) \/[^\/]+$/) {chdir $;
                        Close (PARENT_WR);
                        Close (STDIN); #fcntl (child_rd, F_DUPFD, 0);
                        Syscall (&sys_dup2, Fileno (CHILD_RD), 0);
                        #open (STDIN, "<&child_rd");
                        EXEC ($req _params{script_filename});
                Die ("exec failed");
                } else {print ("content-type:text/plain\r\n\r\n");
            print ' Error:no such CGI app-$req _params{script_filename} may not exist ' not ' executable by this process.\n '; }
        }
}

After the creation is complete, you need to give fcgi Execute permissions: