Nginx Log Common Analysis command summary

First, the concept

Number of concurrent connections

The client initiates a request to the server and establishes a TCP connection. The total number of TCP servers linked per second, which is the number of concurrent connections.

Number of requests

The number of requests means that the client sends Get/post/head packets to the HTTP service after the connection is established.

Expansion: After the server returns the result of the request, there are two scenarios:

• 
  

  • The HTTP header contains the close Word and closes the TCP connection;

  • The HTTP header contains the keep-alive word, this connection is not closed, you can continue to continue to send requests to the HTTP service through the connection, to reduce the number of TCP concurrent connections.

PV (Page view)

The number of visits, that is, page views or clicks, is recorded 1 times each time the user accesses the site. User multiple access to the same page, the amount of access accumulated

UV (Unique visitor)

Number of independent visitors. Each individual device (based on a cookie) is treated as a visitor, and the number of visitors (00:00-24:00) within a day. Access to the same cookie within one day is calculated 1 times.

Standalone IP

The same IP address is computed only once in the 00:00-24:00

Second, nginx configuration

Version

Nginx version:nginx/1.10.2

Log Configuration entry

Access_log/var/log/access.log access;

Log format

Log_format access ' $remote _addr-$remote _user [$time _local] "$request" $request _body ' ' $upstream _addr $upstream _response_time $request _time ' ' $status $body _bytes_sent ' $http _referer ' ' "$http _user_agent" $http _x_forwarded_for ';

Iii. Common Log Analysis commands

1. Total number of requests

Wc-l access.log |awk ' {print $} '

2. Independent IP number

awk ' {print '} ' access.log|sort |uniq |wc-l

3. Number of client requests per second TOP5

Awk-f ' [[] ' {print $} ' access.log|sort|uniq-c|sort-rn|head-5

4. Most frequently accessed IP Top5

awk ' {print '} ' Access.log|sort |uniq-c | Sort-rn |head-5

5. The most frequently visited URL TOP5

awk ' {print $7} ' Access.log|sort |uniq-c | Sort-rn |head-5

6, response to a URL greater than 10 seconds TOP5

awk ' {if ($ >) {print $7}} ' Access.log|sort|uniq-c|sort-rn |head-5

7, HTTP status code (not 200) statistics TOP5

awk ' {if ($13! =) {print $13}} ' access.log|sort|uniq-c|sort-rn|head-5

8. Analysis of the behavior of the source IP with the number of requests greater than 50000

awk  ' {print $1} '  access.log|sort |uniq -c  |sort -rn|awk  ' {if  ($1 > 50000) {print $2}} '  > tmp.txtfor i  in $ (cat tmp.txt) do   echo  $i   >> analysis.txt    echo  "Access Behavior Statistics"  >> analysis.txt   grep  $i    access.log|awk  ' {print $6} '  |sort |uniq -c | sort -rn |head  -5 >> analysis.txt   echo  "Access Interface Statistics"  >> analysis.txt    grep  $i   access.log|awk  ' {print $7} '  |sort |uniq -c |  sort -rn |head -5 >> analysis.txt   echo -e  "\ n"   >> /root/analysis/$Ydate. Txtdone 

Note: If the source IP is from a proxy server, you should change the first command filter address to $http _x_forwarded_for address

awk ' {print $NF} ' access.log|sort |uniq-c |sort-rn|awk ' {if ($ > 50000) {print $}} ' > Tmp.txt

Iv. extension

Nginx Log can use the above command to draw daily website traffic analysis after cutting, then can write a Python script read and send. You can also import data into MySQL and use Python to draw the curve. Of course, the use of log analysis system, such as elk, better results.

This article is from the "Mud" blog, please be sure to keep this source http://cangzihu.blog.51cto.com/6671848/1886640

Nginx Log Common Analysis command summary