Python Crawl Page __python

In recent days there is a net loan blacklist (http://www.p2pblack.com) site is very hot, just have to learn Python's intention, so take this site practicing, crawl the site published "Old Lai information."

Novice learning, a lot of places write bad, record, by the way to ask the Great God to guide the regular expression of the writing, I wrote the regular expression is too complex, feeling should have a simpler way to achieve.

The code is as follows:

#-*-coding:utf-8-*-Import urllib2 import urllib import re class p2pblack:def __init__ (self): #默认每页显示 
        20 Records self.showcount=20 #网站根目录, used to splice the acquired URL self.basepath= "http://www.p2pblack.com" #列表页url
    Self.listurl= "Http://www.p2pblack.com/cheat/frontDeadBeatList.html?currentPage=1&showCount=" #获取列表页 def gethtml (Self,count): Url=self.listurl+str (count) req=urllib2.
        Request (URL) response=urllib2.urlopen (req) page=response.read () Unicodepage=page.decode ("Utf-8") Return unicodepage #获取总条数, add URL to facilitate direct query of all Def GetCount (self): Unicodepage = self.gethtml (self.
        Showcount); Count=re.findall (' <font color=red> (. *?) </font> ', Unicodepage, re.         S) self.showcount=count[0] return self.showcount #获取所有记录的详情url def getallurl (self): #         Count=self.getcount () # Dif=int (count) -1692 # if dif>0: #    COUNT=DIF # unicodepage=self.gethtml (count) unicodepage=self.gethtml urllist = Re.findall ( ' <a href=\ ' (/cheat/showdetail.html.*?) \ ".*?> ', Unicodepage,re." S|re. I) return urllist #获取详情页 def getdetail (self,url): Req=urllib2.
        Request (URL) response=urllib2.urlopen (req) page=response.read () Unicodepage=page.decode ("Utf-8")
        Return unicodepage #遍历所有记录的详情 and go to detail page crawl data def FORRECODR (self): Reclist=self.getallurl () 
            For I in range (len (reclist)): Detailurl=self.basepath+reclist[i] Detail=self.getdetail (Detailurl) #取出编号等信息id相关信息 detidp=re.findall (' <p.*?class= ft_publick_bh ') > (. *?) </p> ', detail, re. S) Detid=re.findall (' .*?<span> (. *?) </span>.*?<span.*?class= "Ft_publick_jicu" > (. *?) </span>.*? ', Detidp[0],re. S) #取出欠债人基本信息 detinfodiv=re.findall (' <div.*?class= "ft_publick_pZxxright ft_publick_pzxxrightss "> (. *?) </div> ', detail, re. S) Detinfo=re.findall (' <p>.*?<span> (. *?) </span>.*?<span> (. *?) </span>.*?</p> ', Detinfodiv[0], re.
            
            S) #打印信息 for I in range (len (detid)): Print detid[i][0]+detid[i][1]
                    For I in range (len (detinfo)): #最后一行记录为照片 if I==len (detinfo)-1: If detinfo[i][1]: Imgurl=re.findall ('  
 In the extraction of numbers and the basic information of the debtor, regular expression I used two times, a crawl I tried several times to achieve the effect is not good, but also hope that the great God can guide. Now the code just grabs the records and prints them, and then the code that is stored in the database is studied. Probably looked, on the internet about the Windows Python installation mysqldb Tutorial write the basic is very messy.