Traffic classification statistics (QQ, WeChat, Weibo, webpages, website apps, and others), access apps

Traffic classification statistics (QQ, Weibo, webpages, website apps, and others) and traffic apps

I was just preparing to press the keyboard. I suddenly remembered that today is Friday. I was a little excited. After a week, I had a good time and can have a rest .. I seem to be writing a Java blog. How can I write a diary? Well, let's get down to the truth.

I wonder if you have ever encountered such a requirement: Count the page views of an article and analyze the details, that is, the access rate of QQ, Weibo, webpages, website apps, and others, our company encountered such a requirement some time ago, so I will explain its implementation.

First, you need to measure the access volume of an article. The access volume here is the precise access volume (the number of visits made by login users is counted as one person multiple times, and the visitor counts as one person based on the ip address ), the following code is used to obtain the ip Address:

Public static String getIpAddr (Request request) {/* String ip = request. getHeader ("x-forwarded-for"); if (ip = null | ip. length () = 0 | "unknown ". equalsIgnoreCase (ip) {ip = request. getHeader ("Proxy-Client-IP");} if (ip = null | ip. length () = 0 | "unknown ". equalsIgnoreCase (ip) {ip = request. getHeader ("WL-Proxy-Client-IP");} if (ip = null | ip. length () = 0 | "unk Nown ". equalsIgnoreCase (ip) {ip = request. getRemoteAddr ();} if (ip. contains (",") {ip = ip. substring (0, ip. indexOf (",");} return ip; */String ipAddress = null; // ipAddress = this. getRequest (). getRemoteAddr (); ipAddress = request. getHeader ("x-forwarded-for"); if (ipAddress = null | ipAddress. length () = 0 | "unknown ". equalsIgnoreCase (ipAddress) {ipAddress = request. getHeader ("Proxy-Clien T-IP ");} if (ipAddress = null | ipAddress. length () = 0 | "unknown ". equalsIgnoreCase (ipAddress) {ipAddress = request. getHeader ("WL-Proxy-Client-IP");} if (ipAddress = null | ipAddress. length () = 0 | "unknown ". equalsIgnoreCase (ipAddress) {ipAddress = request. remoteAddress (); if (ipAddress. equals ("127.0.0.1") {// Based on the NIC, obtain the IP address InetAddress inet = null configured on the local machine. try {inet = InetAddress. getLocalHo St ();} catch (Exception e) {e. printStackTrace ();} ipAddress = inet. getHostAddress () ;}}// when multiple proxies are used, the first IP address is the real IP address of the client, and multiple IP addresses are separated by ',' if (ipAddress! = Null & ipAddress. length ()> 15 ){//"***. ***. ***. ***". length () = 15 if (ipAddress. indexOf (",")> 0) {ipAddress = ipAddress. substring (0, ipAddress. indexOf (",") ;}} return ipAddress ;}

　　

Step 2: insert the access source when saving each access record, for example, qq or so. The following code is used to obtain the access Source:

/*** Obtain the browser type, 0-webpage, 1; // Weibo 2; // qq 5; // 6; // APP 7; // others * @ author LongJin * @ return */public static int getfronm( String client) {Http. context context = ctx (); Http. request request = context. request (); String head = request. getHeader ("user-agent "). toLowerCase (); play.Logger.info ("head =" + request. getHeader ("user-agent"); int from = 0; // pc-side boolean isPc = HttpRequestDeviceUtils. isPcDevice (request) ; // Determine whether the source is pc-side if (! IsPc) {if (head. contains ("weibo") {from = 1; // weibo} else if (getQq (head) {// qq independently writes a judgment method from = 2; // qq} else if (head. contains ("micromessenger") {from = 5; //} else if (StringUtils. isNotBlank (client) {from = 6; // APP} else {from = 7; // others} return from ;}

Determine whether the source is QQ:

/*** Obtain QQ ** @ param str * @ return */public static boolean getQqCount (String str) {Pattern p = Pattern. compile ("\ bqq \ B", Pattern. CASE_INSENSITIVE); Matcher m = p. matcher (str); boolean isQq = false; while (m. find () {isQq = true;} return isQq ;}

The HttpRequestDeviceUtils. isPcDevice (request) checks whether the source pc is implemented as follows:

/** Unique description of Wap gateway Via header */private static String mobileGateWayHeaders [] = new String [] {"ZXWAP ", // The via information of the wap Gateway provided by ZTE, for example, Via = ZXWAP GateWayZTE Technologies, "chinamobile.com", // China Mobile's Nokia wap Gateway, for example: via = WTP/1.1 GDSZ-PB-GW003-WAP07.gd.chinamobile.com (Nokia WAP Gateway 4.1 CD1/ECD13_D/4.1.04) "monternet.com", // mobile dream Gateway, e.g.: Via = WTP/1.1 BJBJ-PS-WAP1-GW08.bj1.monternet.com. (Nokia WAP Gateway 4.1 CD1/ECD13_E/4.1.05) "InfoX", // wap Gateway provided by Huawei, for example, Via = HTTP/1.1 GDGZ-PS-GW011-WAP2 (infoX-WISG Huawei Technologies ), or Via = infoX WAP Gateway V300R001 Huawei Technologies "XMS 724 Solutions HTG", // the wap Gateway of a foreign telecom operator. I don't know which one is "wap.lizongbo.com ", // The simulated header information "Bytemobile" during self-testing. // It seems to be a solution provided to the mobile Internet to improve network operation efficiency. For example: via = 1.1 Bytemobile OSN WebProxy/5.1 }; /** User-Agent keywords such as IE or Firefox on the computer */private static String [] pcHeaders = new String [] {"Windows 98 "," Windows ME "," Windows 2000 "," Windows XP "," Windows NT "," Ubuntu "," Macintosh "," Linux "}; /** keywords in the User-Agent of the mobile browser */private static String [] mobileUserAgents = new String [] {"Nokia", // Nokia, this is also written on a cottage machine. It is still a mobile phone. Mozilla/5.0 (Nokia5800 XpressMusic) UC AppleWebkit (like Gecko) Safari/530 "SAMSUNG ", // SamSung mobile SAMSUNG-GT-B7722/1.0 + SHP/VPP/R5 + Dolfin/1.5 + Nextreaming + SMM-MMS/1.2.0 + profile/MIDP-2.1 + configuration/ CLDC-1.1 "MIDP-2", // j2me2.0, Mozilla/5.0 (SymbianOS/9.3; U; Series60/3.2 NokiaE75-1/110.48.125 Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/413 (KHTML like Gecko) safari/413 "CLDC1.1", // M600/MIDP2.0/CLDC1.1/Screen-240X320 "SymbianOS", // Saipan system, "MAUI ", // The default ua "UNTRUSTED/1.0", // The ua of the mtk stockade is suspected to be a stockade. You can determine whether the phone is "Windows CE", // Windows CE, Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; IEMobile 7.11) "I Phone ", // does the iPhone switch to wap? Regardless of it, distinguish it first. Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; zh-cn) AppleWebKit/532.9 (KHTML like Gecko) Mobile/8B117 "iPad ", // iPad ua, Mozilla/5.0 (iPad; U; cpu OS 3_2 like Mac OS X; zh-cn) AppleWebKit/531.21.10 (KHTML like Gecko) version/4.0.4 Mobile/7B367 Safari/531.21.10 "Android", // does Android convert to wap? Mozilla/5.0 (Linux; U; Android 2.1-update1; zh-cn; XT800 Build/TITA_M2_16.22.7) AppleWebKit/530.17 (KHTML like Gecko) Version/4.0 Mobile Safari/530.17 "BlackBerry ", // BlackBerry8310/2.7.0.106-4.5.0.182 "UCWEB", // does ucweb only provide wap pages? Nokia5800 XpressMusic/UCWEB7.5.0.66/50/999 "ucweb", // The lower-case ucweb looks like the uc Proxy Server Mozilla/6.0 (compatible; MSIE 6.0;) Opera ucweb-squid "BREW ", // very strange ua, such as: REW-Applet/0x20068888 (BREW/3.1.5.20; DeviceId: 40105; Lang: zhcn) ucweb-squid "j2 ", // very strange ua, only the four characters of the characters "YULONG", // YuLong phone, YULONG-CoolpadN68/10.14 IPANEL/2.0 CTC/1.0 "YULONG ", // or Yulong "COOLPAD", // Yulong copi YL-COOLPADS100/08.10.100 POLARIS/2.9 CTC/1.0 "TIANYU", // day Cell phone TIANYU-KTOUCH/V209/MIDP2.0/CLDC1.1/Screen-240X320 "TY-", // days, TY-F6229/701116_6215_V0230 JUPITOR/2.2 CTC/1.0 "K-Touch ", // or Tianyu K-Touch_N2200_CMCC/TBG110022_1223_V0801 MTK/6223 Release/30.07.2008 Browser/WAP2.0 "Haier", // Haier mobile phone, Haier-HG-M217_CMCC/3.0 Release/12.1.2007 Browser/WAP2.0 "DOPOD ", // Lenovo phone "Lenovo", // Lenovo phone, Lenovo-P650WG/100lmp/LML Release/2010.02.22 Profile/MIDP2.0 Configuration/CLDC1.1 "L Enkyo ", // Lenovo mobile phone, such as: LENOVO-P780/176A" HUAQIN ", // Hua Qin mobile phone" AIGO-", // patriot actually also out of the mobile phone, AIGO-800C/2.04 TMSS-BROWSER/1.0.0 CTC/1.0 "CTC/1.0", // meaning unknown "CTC/2.0", // meaning unknown "CMCC", // mobile custom phone, k-Touch_N2200_CMCC/TBG110022_1223_V0801 MTK/6223 Release/30.07.2008 Browser/WAP2.0 "DAXIAN", // DAXIAN X180 UP. browser/6.2.3.2 (GUI) MMP/2.0 "MOT-", // Motorola, MOT-MOTOROKRE6/1.0 LinuxOS/2.4.20 Release/8.4.2006 Browser/Opera8.00 Profile/MID P2.0 Configuration/CLDC1.1 Software/R533_G_11.10.54R "SonyEricsson", // Sony Ericsson mobile phone, SonyEricssonP990i/R100 Mozilla/4.0 (compatible; MSIE 6.0; Symbian OS; 405) opera 8.65 [zh-CN] "GIONEE", // GIONEE mobile phone "HTC", // HTC mobile phone "ZTE", // ZTE mobile phone, ZTE-A211/pda-a2v1.0.0/WAP2.0 Profile "HUAWEI", // HUAWEI phone, "webOS", // palm phone, Mozilla/5.0 (webOS/1.4.5; U; zh-CN) appleWebKit/532.2 (KHTML like Gecko) Version/1.0 Safari/532.2 Pre/1.0 "GoBrowser ", // 3g GoBrowser. user-Agent = Nokia5230/gobrow./ 2.0.290 Safari "IEMobile", // Windows CE mobile phone comes with a browser, "WAP2.0" // supports wap 2.0 }; /*** determine whether the request is pc * @ param Request * @ return */public static boolean isPcDevice (request) {boolean B = false; boolean pcFlag = false; boolean mobileFlag = false; String via = request. getHeader ("Via"); String userAgent = request. getHeader ("user-agent"); // String xMsisdn = reque St. getHeader ("X-MSISDN");/* Map <String, String []> headers = request. headers (); play.Logger.info ("xM =" + headers); */for (int I = 0; userAgent! = Null &&! UserAgent. trim (). equals ("") & I <pcHeaders. length; I ++) {if (userAgent. contains (pcHeaders [I]) {pcFlag = true; break ;}}for (int I = 0;! = Null &&! Via. trim (). equals ("") & I <mobileGateWayHeaders. length; I ++) {if (. contains (mobileGateWayHeaders [I]) {mobileFlag = true; break ;}} for (int I = 0 ;! MobileFlag & userAgent! = Null &&! UserAgent. trim (). equals ("") & I <mobileUserAgents. length; I ++) {if (userAgent. contains (mobileUserAgents [I]) {mobileFlag = true; break;} if (mobileFlag = false & pcFlag = true) {B = true;} return B ;}

　　

So far, the classification of visits is basically complete. You can combine the statistics of visits to achieve the ranking function of original articles and make a pie chart with the statistical details. It says that a sudden power failure may be due to a holiday, excited.

Ps: I will only explain some of the core parts here, but I will not explain them in detail. In the whole process I implemented, there was a statistical accuracy problem, that is, the same as that of the LAN and the Internet. As a result, only one person was allowed to access the same LAN during traffic statistics, hope you can give me some advice.