Apache Log Segmentation & Log Analysis

Apache Log Split

As the number of visits to the site is increasing, by default the Apache server generates a single log file will be more and more large, if you do not use log segmentation, when the journal file is growing, when taking up too much disk space will be deleted by the system, and as the log file increases, For the administrator of the Apache server Access analysis, network security Monitoring, network health monitoring and other operations brought greater difficulty, so managing these massive logs on the site is significant.

Experiment Description:

• Complete the log segmentation for Apache
• This experiment uses the Apache2.2 version
• Server ip:192.168.100.107
• Services Required: httpd, DNS
• Server System Redhat6.5 Version

Service Deployment

Rpm-q httpd//Check if Apache installs and installs the version
Rpm-q bind//Check whether the DNS service is installed
Service Iptables Stop
Setenforce 0

1. Configure the DNS service

For details see also: DNS Domain name Resolution service configuration

RPM-IVH/MNT/PACKAGES/BIND-9.8.2-0.17.RC1.EL6_4.6.X86_64.RPM//First install DNS domain name Resolution Service
VIM/ETC/NAMED.CONF//Configure domain name resolution master configuration file

options {        listen-on port 53 { 192.168.100.107; }; //监听端口主机IP        listen-on-v6 port 53 { ::1; };        directory       "/var/named";        dump-file       "/var/named/data/cache_dump.db";        statistics-file "/var/named/data/named_stats.txt";        memstatistics-file "/var/named/data/named_mem_stats.txt";        allow-query     { any; };   //允许任意网段主机都可解析        recursion yes;

Vim/etc/named.rfc1912.zones//Edit Zone Profile

zone "yun.com" IN {        type master;        file "yun.com.zone";        allow-update { none; };};

cd/var/named//Switch to the zone data configuration directory
Cp-p named.localhost yun.com.zone//Copy area Data profile template
VIM Yun.com.zone//Configure zone Data profile

$TTL 1D@       IN SOA  @ admin. (     //修改管理员邮箱                                        0       ; serial                                        1D      ; refresh                                        1H      ; retry                                        1W      ; expire                                        3H )    ; minimum        NS      @        A       127.0.0.1www IN  A       192.168.100.107   //设置解析地址

Service named start//Turn on DNS services

2. Configuration Log Segmentation

RPM-IVH/MNT/PACKAGES/HTTPD-2.2.15-29.EL6_4.X86_64.RPM//Installation Apache2.2
vim/etc/httpd/conf/httpd.conf//Edit Master Profile

Listen 192.168.100.107:80  //更改监听地址和监听端口#Listen 80   //注释IPV6的监听...ServerName www.yun.com:80  //打开主机域名

Vim/etc/httpd/con.d/vhost.conf

Write the following content:

<VirtualHost *:80>      //任意网址的80端口   ServerAdmin [email protected]  //管理员邮箱   DocumentRoot "/usr/local/apache/htdocs"  //创建Apache域名首页站点   ServerName www.yun.com   //访问域名   ErrorLog "|/usr/sbin/rotatelogs -l /usr/local/apache/logs/www.benet.com-error_%Y%m%d.log 86400"     //调用Apache自带日志分割工具rotatelogs   //错误日志   CustomLog "|/usr/sbin/rotatelogs -l /usr/local/apache/logs/www.benet.com-access_%Y%m%d.log 86400" combined   //访问日志，按年月日显示   //注：此处需要制定日志分割的存放位置/usr/local/apache/logs</VirtualHost>

Mkdir-p/usr/local/apache/htdocs//Create a site
echo "This is Rotatelogs Web" >/usr/local/apache/htdocs/index.html//Edit Home content
Mkdir-p/usr/local/apache/logs//create log split storage directory

Service httpd start//start httpd Services

When visiting a Site page:

You can see that the log file was generated under/usr/local/apache/logs:

When the server system date is modified, the Apache Service website is accessed again, and a log file is generated by date:

Access Log Analysis

In the httpd log file, a large number of client access information is recorded, through the analysis of this information, you can timely understand the Web site access, here through the installation of Awststs log analysis System, complete automated log analysis and statistical work.

1. Deploy Awstats Analysis System

Mount.cifs//192.168.100.3/awstats/opt/abc/
Mount the Awstats software to/OPT/ABC with shared mounts
Tar zxvf awstats-7.6.tar.gz-c/opt/unpacking Package
Mv/opt/awstats-7.6/usr/local/awstats//Rename the package to/usr/local/because the software can be used directly after decompression, so there is no need to install
Cd/usr/local/awstats/tools//Switch to the tool directory

./awstats_configure.pl//Execute configuration file

Config file path (‘none‘ to skip web server setup):> /etc/httpd/conf/httpd.conf          //输入主配置文件Your web site, virtual server or profile name:> www.yun.com                       //输入域名//其他全部是y 或者 回车

vim/etc/httpd/conf/httpd.conf//When the above file is executed, open the HTTPD service master configuration file again, and find the following more at the end

<Directory "/usr/local/awstats/wwwroot">    //awstats默认站点    Options None    AllowOverride None    Order allow,deny    Allow from all</Directory>

vim/etc/awstats/awstats.www.yun.com.conf//Edit the profile of the Web site that needs to be counted

LogFile="/var/log/httpd/access_log"    //修改访问日志文件位置//也就是指向日志分割后的日志文件所在位置DirData="/var/lib/awstats"              //用来指定数据目录，可采用默认，awstats 默认不存在（需要创建）

Mkdir-p/var/lib/awstats//create default data directory
Service httpd restart//restart HTTPD services

Access in the browser:
Http://www.yun.com/awstats/awstats.pl?config=www.yun.com to access the statistics page

cd/usr/local/awstats/tools/
./awstats_updateall.pl Now//Refresh statistics

2. Optimize page address

According to the above method, when accessing the statistics page, the page address is too long and error-prone, so the page address needs to be optimized for easy management and access.
Cd/usr/local/apache/htdocs//Switch to Web Access under Default site
Vim awb.html//SET STATISTICS page Home

Replace the statistics page URL with www.yun.com/awb.html

CRONTAB-E//Refresh statistics with recurring tasks

*/5 * * * *  /usr/local/awstats/tools/awstats_updateall.pl now   //设置每隔五分钟刷新

Service httpd restart//restart HTTPD services

Enter www.yun.com/awb.html in the browser for access

Apache Log Segmentation & Log Analysis