Apache log analysis and system CPU, memory, load monitoring

1. According to the Apache log, get the maximum number of concurrent records in the second day

tail-10000./demo.fdipzone.com_access_log.2014-01-16_01 | Cut-d "["-F 2 | awk ' {print $} ' | Sort | uniq-c | Sort-nr | Head

tail-10000./demo.fdipzone.com_access_log.2014-01-16_01 | Awk-f "[" ' {print $;} ' | awk ' {print $} ' | Sort | uniq-c | Sort-nr | Head

Run Result:

fdipzone@ubuntu:~$ tail-100000./demo.fdipzone.com_access_log.2014-01-16_01 | Cut-d "["-F 2 | awk ' {print $} ' | Sort | uniq-c | Sort-nr | Head   
    109 16/jan/2014:22:51:16  
     16/jan/2014:22:46:56  
     16/jan/2014:23:56:30  
     16/jan/ 2014:22:48:27  
     16/jan/2014:23:55:52  
     16/jan/2014:23:51:29  
     16/jan/2014:23:00:33  
     16/jan/ 2014:23:28:04  
     16/jan/2014:23:17:35  
     16/jan/2014:23:05:38

tail-10000 is the maximum number of rows for access log, please use tail-500000, depending on the actual settings, such as the maximum number of rows in the access log is 500000.

Head defaults to display the first 10 lines, and if you want to modify, you can add the-n parameter, for example, Head-n 100 will display the first 100 lines

More queries

Maximum access per minute  
awk ' {print $} ' Access_log |cut-c 14-18|sort|uniq-c|sort-nr|head the  
      
highest hourly access number  
awk ' {print $} ' acces S_log |cut-c 14-15|sort|uniq-c|sort-nr|head-n  
      
Specifies the number of accesses per second of a minute  
grep ' 01/nov/2013:15:59 ' access.log | cut-d ' ["- F 2 | awk ' {print $} ' | Sort | uniq-c | Sort-nr | Head-60

2. Record CPU and memory usage to log

The Vmstat command is the most common Linux/unix monitoring tool that shows the state value of a server at a given time interval, including CPU utilization, memory usage, virtual memory exchange, and IO reading and writing.

Vmstat a total of two parameters, one is the interval of each collection, the other is the number of collection.

The Vmstat 1 11 indicates that each collection time is 1 seconds, and a total of 11 times are collected. Because the first collection of Vmstat records is not correct, so if you want to collect 10 times, please set the number of times 11, so that you can get 10 useful records.

fdipzone@ubuntu:~$ Vmstat 1 procs-----------memory-------------Swap-------io-----system------CPU----r b    SWPD free buff cache si so bi bo in CS us sy ID WA 0 0 0 335240 52072 241320 0 0 689  42 234 460 4 13 80 3 1 0 0 335860 52072 241320 0 0 0 0 352 528 2 2 97 0 2 the 0 0     336364 52072 241320 0 0 0 0 564 692 4 3 94 0 0 0 0 336992 52072 241320 0 0-0 0 316 483 1 2 97 0 2 0 0 336432 52076 241476 0 0 156 0 332 554 2 6 91 2 0 0 0 336 928 52084 241480 0 0 0 84 349 494 1 3 96 0 0 0 0 337564 52084 241476 0 0 0 0-2  63 437 0 2 98 0 0 0 0 338068 52084 241476 0 0 0 0 299 456 1 2 97 0 0 0 The 0 338696  52084 241476 0 0 0 0 321 473 1 3 96 0 1 0 0 339200 52084 241476 0 0 0 0-271 461 0 2 98 0 0 0     0 339828 52084 241476 0 0 0 0 255 429 1 2 96-0 

Vmstat 1 >> vmstat.log per second, written to Vmstat.log

Vmstat output for each parameter description:

R indicates that the running queue (that is, how many processes are actually allocated to the CPU), when this value exceeds the number of CPUs, there will be a CPU bottleneck. This also and top of the load is related to the general load more than 3 higher, more than 5 on the high, more than 10 is not normal, the state of the server is very dangerous. The top load is similar to the run queue per second. If the running queue is too large, which means your CPU is busy, it will generally cause high CPU usage.

b represents a blocked process.

SWPD virtual memory has been used in size, if greater than 0, indicates that your machine physical memory is not enough, if not the reason for the program memory leak, then you should upgrade the memory or the memory of the task to migrate to other machines.

Free of the physical memory size, my test machine memory total 1G, remaining 330M.

Buff Linux/unix system is used to store, directory contains what content, permissions, etc. cache, I this machine probably occupy more than 500 m

Cache cache is directly used to memorize the files we open, to buffer the files, (here is the smart of Linux/unix, the free physical memory part of the cache for files and directories, is to improve the performance of program execution, when the program uses memory, buffer/ Cached will soon be used. )

Si per second the size of the virtual memory from disk, if this value is greater than 0, indicates that the physical memory is not enough or memory leaks, to find the memory process to solve the loss.

So per second virtual memory is written to disk size, if this value is greater than 0, ditto.

The number of blocks received by the BI block device per second, where the block device refers to all the disks and other block devices on the system, and the default block size is 1024byte.

The number of blocks sent by the Bo block device per second, for example, when we read the file, Bo must be greater than 0. Bi and Bo are generally close to 0, or IO is too frequent and need to be adjusted.

The number of CPU interrupts per second, including time outages

Back to the column page: http://www.bianceng.cnhttp://www.bianceng.cn/Servers/web/