Apache server Log analysis application instance

With the rapid development of the Internet, the analysis of website logs is particularly important for traditional enterprise websites and Internet enterprises to understand their customer sources, click rates, and other information, I would like to share with you the log analysis system I have made and hope to help you.

You are welcome to copy this article, but please keep it complete. Thank you !!

I. software and tools used
1, Apache server official site: http://www.apache.org
2, awstats log analysis software Official Website: http://awstats.sourceforge.net/
3, cronolog capture software Official Website: http://www.cronolo.org
II. Environment Architecture
1, RedHat AS 3
2. Apache
./Configure-prefix =/usr/local/apache2/-- enable-so
Make
Make install
3, awstats
Directly tar to/usr/local/awstats
4. cronolog
./Configure-prefix =/usr/local/cronolog
Make
Make install
3. Configuration
1. apache
A. Generate logs for a VM in apache:

Global Environment variable settings:
Code:
LogFormat "% h % l % u % t" % r "%> s % B" % {Referer} I "" % {User-Agent} I "combined
SetEnvIf Request_URI. gif $ gif-image
SetEnvIf Request_URI. GIF $ gif-image
SetEnvIf Request_URI. jpg $ gif-image
SetEnvIf Request_URI. JPG $ gif-image
SetEnvIf Request_URI. png $ gif-image
SetEnvIf Request_URI. swf $ gif-image
SetEnvIf Request_URI. SWF $ gif-image
SetEnvIf Request_URI. css $ gif-image
SetEnvIf Request_URI. CSS $ gif-image
SetEnvIf Request_URI. js $ gif-image
SetEnvIf Request_URI. JS $ gif-image
SetEnvIf Request_URI. ico $ gif-image
 

Logs are generated, but images are not included.
VM Log Settings
Code:
<VirtualHost 12.34.56.78: 80>
ServerAdmin webmaster@abc.net
DocumentRoot/db/htdocs/www
ServerName _ blank> www.abc.net
CustomLog "|/usr/local/cronolog/sbin/cronolog/db/logs/www. % Y-% m-% d. log" combined env =! Gif-image
# Use cronolog to capture apache log files generated by date stored in/db/logs/
</VirtualHost>

Note: log files are generated for each virtual host. Therefore, you can configure Log Analysis for multiple virtual hosts on one server.
B. Alias settings
Code:

#
# Directives to allow use of AWStats as a CGI
#
Alias/awstatsclasses "/usr/local/awstats/wwwroot/classes /"
Alias/awstatscss "/usr/local/awstats/wwwroot/css /"
Alias/awstatsicons "/usr/local/awstats/wwwroot/icon /"
ScriptAlias/awstats/"/usr/local/awstats/wwwroot/cgi-bin /"

#
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats/wwwroot">
Options None
AllowOverride None
Order allow, deny
Allow from all
</Directory>
 

This configuration is automatically generated after running awstats/tools/awstats_configure.pl, so you do not need to add it manually.
C. awstats requires cgi support. If cgi is not added, add it now.
Code:

./Apxs-c-I/usr/local/src/httpd-2.0.50/modules/generators/mod_cgi.c
 

Add it in httpd. conf.
Code:

LoadModule cgi_module modules/mod_cgi.so
 

D. Some directories must be authenticated to protect log statistics.
Code:

./Apxs-c-I/usr/local/src/httpd-2.0.50/.../mod_auth.c
 

Add it in httpd. conf.
Code:

LoadModule auth_module modules/mod_auth.so
 

Configure the directory to be authenticated:
Code:

<Directory "/db/htdocs/www/awstat/">
AuthType Basic
AuthName "Restricted Files"
AuthUserFile/usr/local/apache2/passwd/passwords
Require user loguser
</Directory>
 

The passwords password file is generated under/usr/local/apache2/passwd /.
Run the following code to generate a loguser user and its password.
Code:

/Usr/local/apache2/bin/htpasswd-c/usr/local/apache2/passwd/passwords loguser

Note: you must first create the passwd directory under/usr/local/apache2 /.
2, Awstats
Run/usr/local/awstats/tools/awstats_configure.pl
The main setting is to enter the configuration file name of the host name for statistics, such as www.abc.net.
Next, modify the awstats configuration file generated by the configuration file. The default location is:
/Etc/awstats/awstats.www.abc.net. con mainly specifies the log location
Code:

LogFile = "/db/logs/www. % YYYY-4-% MM-2-% DD-2.log"
 

Create directory: mkdir/var/lib/awstats/
Run:
Code:

/Usr/local/awstats/wwwroot/cgi-bin/awstats. pl-update-config = www.abc.net
 

Update, after update, you can enter _ blank> http://www.abc.net/awstats/awstats.pl in the browser to access.

3. cronolog (no need to modify it)
4. crontab Modification
Add the following command in crontab to ensure that the update is performed every 15 minutes. You can also modify the update interval as needed.
Code:

*/15 */usr/local/awstats/wwwroot/cgi-bin/awstats. pl-update-config = www.abc.net
 

Iv. security considerations
1. The above settings also add security risks due to cgi support. You can use awstats's awstats_buildstaticpages.pl to generate static pages and add them to crontab.
Code:

*/15 */usr/local/awstats/tools/awstats_buildstaticpages.pl-update-config = www.abc.net-lang = cn-dir =/db/htdocs/www/awstat /- awstatsprog =/usr/local/awstats/wwwroot/cgi-bin/awstats. pl
 

In this case, various static pages are generated at/db/htdocs/www/awstat/in less than 15 minutes.

2. Protect/db/htdocs/www/awstat/from unauthorized user access

Add in httpd. conf
Code:

<Directory "/db/htdocs/www/awstat/">
AuthType Basic
AuthName "Restricted Files"
AuthUserFile/usr/local/apache2/passwd/passwords
Require user loguser
</Directory>
 

If the Home Directory of the www.abc.net VM is/db/htdocs/www/, enter html "> http://www.abc.net/awstat/awstats.www.abc.net.html in the browser to access the static page carefully
3. Do not forget to comment out static pages.
LoadModule cgi_module modules/mod_cgi.so

5. To my favorite Feifei.

6, because I am a beginner, this article is slightly rough, if you have any guidance and questions welcome to contact me, we exchange to improve: hoash2003@hotmail.com

You are welcome to copy this article, but please keep it complete. Thank you !!