NGINX: Statistics Website of PV, UV, Independent IP

Do the site know, usually have to query the site PV, UV and other Web site access data, of course, if the site did a CDN, Nginx local log is meaningless, the following on the Nginx website log access data to do the following statistics;

Concept:

• UV (Unique Visitor): Independent visitors, each individual Internet PC (based on a cookie) as a visitor, within a day (00:00-24:00), access to the number of visitors to your site. access to the same cookie within one day is calculated 1 times
• PV (Page view): Traffic , that is, page views or clicks , users each visit to the site is recorded 1 times. User multiple access to the same page, the amount of access accumulated
• Statistics independent IP:00:00-24:00 within the same IP address is only counted once, do the site optimization of the friend most concerned about this

First declare the environment, the operation of Nginx version 1.7, back-end Tomcat is running a dynamic interactive program (user authentication required, if the static page will not catch the cache value, $http _cookie is null), that is;

Nginx Log File configuration

HTTP {include mime.types;    Default_type Application/octet-stream; Log_format Main ' $remote _addr-[$time _local] "$request" "-$status" User_cookie: $guid "';    #User_Cookie为日志显示字符, $guid is a variable, the specific content is defined below, can also be written in the log format $http_cookie display full cookie content <br> sendfile on;        Keepalive_timeout 65;        Upstream Backserver {ip_hash;        Server 1.1.2.2:8080; Server 1.1.2.3:8080;}        server {Listen 80;       server_name localhost;        #if ($http _cookie ~* "(. *) $") matches all content if ($http _cookie ~* "csid= ([a-z0-9]*)") {set $guid $;         } #只匹配CSID字符信息, here is the regular expression <br> access_log Logs/host.access.log main;               Location ~* ^ (. *) $ {#limit_req zone=allips burst=1 nodelay;             Proxy_pass Http://backserver;             Proxy_set_header Host $host;             Proxy_set_header X-real-ip $remote _addr;          Proxy_set_header remote-host $remote _addr;   Proxy_set_header x-forwarded-for $proxy _add_x_forwarded_for;             Client_max_body_size 8m;        } error_page 502 503 504/50x.html;        Location =/50x.html {root html; }}

Note: $http _cookie This value inside is a value of a cookie, in the middle with ";" Separated

Log Output format

192.168.40.2-[02/nov/2016:15:44:35 +0800] "get/wcm/app/main/refresh.jsp?r=1478072325778 http/1.1"-"User_cookie" : 7f00000122a5597c46607b1c0a7ec016 "192.168.40.2-[02/nov/2016:15:44:35 +0800]" get/webpic/w0201611/w020161102/ W020161102566715167404.jpg http/1.1 "-user_cookie:7f00000122a5597c46607b1c0a7ec016" 119.255.31.109-[02/Nov/ 2016:15:44:36 +0800] "get/wcm/app/main/refresh.jsp?r=1478072510132 http/1.1"-"User_cookie : 7f000001237921be9237838aec65704d "119.255.31.109-[02/nov/2016:15:44:36 +0800]" get/wcm/app/message/message_query _service.jsp? READFLAG=0&MSGTYPES=1%2C2%2C3 http/1.1 "-user_cookie:7f000001237921be9237838aec65704d" 192.168.40.2-[02/ nov/2016:15:44:37 +0800] "get/wcm/app/message/message_query_service.jsp? READFLAG=0&MSGTYPES=1%2C2%2C3 http/1.1 "-user_cookie:7f00000123d3bf2345115eaac21f71e0" 192.168.40.2-[02/ nov/2016:15:44:37 +0800] "get/wcm/app/message/message_query_service.jsp?  READFLAG=0&MSGTYPES=1%2C2%2C3 http/1.1 "-"user_cookie:7f00000123ef73896df98eda9950944e" 192.168.40.2-[02/nov/2016:15:44:37 +0800] "get/wcm/app/message /message_query_service.jsp? READFLAG=0&MSGTYPES=1%2C2%2C3 http/1.1 "-user_cookie:7f00000123fe0f9c397e1a8f0c4f044b" 192.168.40.2-[02/ nov/2016:15:44:37 +0800] "get/wcm/app/main/refresh.jsp?r=1478072511427 http/1.1"-"User_cookie : 7f00000123a465b7ea1de0af0ae671b7 "119.255.31.109-[02/nov/2016:15:44:38 +0800]" get/wcm/app/message/message_query _service.jsp? READFLAG=0&MSGTYPES=1%2C2%2C3 http/1.1 "-user_cookie:7f00000123d89b11302df80ae773c900"

PV statistics can count individual link address visits: [[email protected] logs]# grep index.shtml host.access.log | Wc-l Total PV Volume: [[email protected] logs]# awk ' {print $6} ' Host.access.log | Wc-l standalone IP [[email protected] logs]# awk ' {print $} ' Host.access.log | Sort-r |uniq-c | WC-LUV statistics [[email protected] logs]# awk ' {print $} ' Host.access.log | Sort-r |uniq-c |wc-l

Cookie test Page

For a cookie, you can use the following HTML code, edit, add the cookie that needs to be planted

#index. html <! DOCTYPE HTML PUBLIC "-//W3C//DTD XHTML 1.0 transitional//en" "Http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd ">
NGINX: Statistics Website of PV, UV, Independent IP