Python Crawler Learning Note 1

Joined the lab, to write crawlers, previously written in Java, the newly learned Python practice practiced hand

The first is the awarding process, the framework is as follows:

The first is the post method, the code is as follows:

1 ImportUrllib2 ImportUrllib23Url='http://someserver.com/cgi-bin/register.cgi'4User_agent='mozilla/4.0'5values={6     'name':'liuyuqing',7     ' Location':'DUT',8     'language':'python'9 }Tenheaders={ One     'Cookies':'name=liuyuqing', A     'user-agent': User_agent - } -Data=Urllib.urlencode (values) thereq=Urllib2. Request (url,data,headers) -Response=Urllib2.urlopen (req) -The_page=response.read ()

If the request does not have a data parameter, the Get method is sent

Response.geturl () #获得真正的url

Response.info () #获得返回报文头

There are also some custom opener and handler knowledge for the moment, do not learn to start the regular expression ~

=================== Gorgeous split-line ===========================

The basic knowledge of regular expression is not a table, directly speaking Python application:

Import Rereg=re.compile (R'<title> (. *) </title>') List=  Re.findall (reg,text) for in list:    print  i      Pass

The most basic HelloWorld so much, the rest of tomorrow, sleep the zzz ...

Python Crawler Learning Note 1