Python Pycurl Library--Capture packet analysis for Web site

students who often use the keynote network may feel good about the performance of the keynote network for page elements. It can do a detailed detection of a URL, including: Blocking time, DNS resolution time, establish connection time, SSL handshake time, make request time, first packet time and so on.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/45/D9/wKiom1Psec_iceBrAAGpCF7Ygqw296.jpg "title=" Jidiao.jpg "width=" "height=" 193 "border=" 0 "hspace=" 0 "vspace=" 0 "style=" width:500px;height:193px; "alt=" Wkiom1psec_icebraagpcf7ygqw296.jpg "/>

In fact, we can do it. For example, the Python Pycurl library can be used to collect data, which can then be written to Redis or MySQL for the collected data. Finally, the front end is displayed in graphical form using Echars.

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/45/DB/wKioL1PsfN7Dk1JqAAGGmOAYRYc068.jpg "style=" float: none; "title=" ec1.jpg "alt=" Wkiol1psfn7dk1jqaaggmoayryc068.jpg "/>

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/45/D9/wKiom1Pse8bg6o5mAADIBgSmR6g461.jpg "style=" float: none; "title=" ec2.jpg "alt=" Wkiom1pse8bg6o5maadibgsmr6g461.jpg "/>

Echars is an open source project of Baidu, the function is very powerful (project url:http://echarts.baidu.com) can show the data through the form of various graphs.

Use Pycurl to monitor a specified site phase relationship to

Pycurl is a Python library implemented in C, Pycurl can be used to get a response object for a specified URL. It supports a number of protocols, such as: FTP, FTPS, HTTP, HTTPS, Telnet, and so on.

You can use a script written by the Pycurl Library to place on a CDN node in a different region, test the specified site with the Crond job or daemon, and then send the test results via RFC or HTTP to the server to write to Redis or MySQL. Finally, through the front page display.

#!/usr/bin/env pythonimport pycurlimport sysimport jsonweb_sites = sys.argv[1] Def main ():     c = pycurl. Curl ()     c.setopt (Pycurl. Url, web_sites)                                  # Set Session    c.setopt (Pycurl. followlocation, 1)     content = c.perform ()                                                #执行传输动作     dns_time = c.getinfo (pycurl. Namelookup_time)                       #DNS   Build Time     conn_time = c.getinfo (Pycurl. Connect_time)                         #TCP/ip  three time handshake     starttransfer_time =  C.getinfo (Pycurl. Starttransfer_time)         #首包时间     total_time =  c.getinfo (Pycurl. Total_time)                          #上一请求的总时间     http_code = c.getinfo ( Pycurl. Http_code)                            #HTTP响应代码     redirect_count =  c.getinfo (Pycurl. Redirect_count)                 #重定向次数     size_upload = c.getinfo (pycurl. Size_upload)                        #上传字节大小     size_download = c.getinfo (pycurl. Size_download)                    #下载字节大小     header_size = c.getinfo (Pycurl. Header_size)                        #头部大小     request_size = c.getinfo (pycurl. Request_size)                      #请求大小     content_type = c.getinfo (pycurl. Content_Type)                     #请求内容类型     c.close ()                                                           #关闭处理Curl的session      data = json.dumps ({' Dns_time ':d ns_time,                                  ' Conn_time ':conn_time,                                 ' Starttransfer_time ':starttransfer_time,                            ' Total_time ':total_time,                         ' Http_code ':http_code,                        ' Redirect_count ':redirect_count,                        ' Size_upload ':size_upload,                        ' Size_download ':size_download,                        ' Header_ Size ':header_size,                        ' Request_size ':request_size,                        ' Content_Type ': Content_Type})      return dataif __name__ ==  "__main__":    print  Main ()

The result of the script execution is as follows:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/45/DC/wKioL1Psg5OCHEy1AAFTfFKX0cY435.jpg "title=" Code.jpg "alt=" Wkiol1psg5ochey1aaftffkx0cy435.jpg "/>

In addition to some of the appropriate information used in the above script, Pycurl supports a number of response messages:

Pycurl. Pretransfer_time #建立连接后到开始传输的时间

Pycurl. Redirect_time #如果存在跳转, the time spent

Pycurl. Speed_upload #上传速度

Pycurl. Content_length_download #下载内容长度

Pycurl. Content_length_upload #上传内容的长度

Pycurl. Speed_download #下载速度

Pycurl.info_filetime #文件的时间信息

Pycurl.http_connectcode #HTTP连接代码

In this way, the cache nodes in each region can run, and the acquired data can be uploaded to the server via RFC or other means. Finally, the front page rendering is OK. Because our unit bought the tone of service, so I did not write the front-end page, if any need, you can do a bit.

This article is from the "David" blog, so be sure to keep this source http://davidbj.blog.51cto.com/4159484/1540195