The introduction of web crawler--Case two: Crawling students ' achievements in the educational administration system

Resources:

Objective of this post:

1. Simulated Landing School educational system

2. Grasping students ' achievements in the educational system

3. Save the crawled content to an Excel table and calculate the average score and performance points

#-*-coding:utf-8-*-"""Created on Sat Apr 18:26:46 2016@author:wuhan"""ImportUrllibImportUrllib2ImportCookielib fromBs4ImportBeautifulSoupImportSYSImportXLWT fromXlrdImportOpen_workbookclassSDU:def __init__(self): Self.loginurl='http://zhjw.dlut.edu.cn/loginAction.do'Self.gradeurl='http://zhjw.dlut.edu.cn/gradeLnAllAction.do?type=ln&oper=fainfo&fajhh=4289'self.cookies=Cookielib. Cookiejar () Self.postdata=Urllib.urlencode ({'Zjh':'*********',        'mm':******'}) Self.opener=Urllib2.build_opener (urllib2. Httpcookieprocessor (self.cookies)) Self.book= XLWT. Workbook (encoding='Utf-8', style_compression=0)defGetPage (self): request=Urllib2. Request (URL=Self.loginurl, Data=self.postdata) Response=Self.opener.open (Request) Response=Self.opener.open (Self.gradeurl)returnResponse.read (). Decode ('GBK')            defGetgrade (self): page=self.getpage () page=beautifulsoup (str (page)) TMP= Page.find_all ('TR', Class_ ="Odd") Sheet= Self.book.add_sheet ('Course Information', cell_overwrite_ok=True) I=0 foreachinchTmp:tag= Each.find_all ('TD', align="Center") J=0 forTdinchtag:p= Td.find ('P', align="Center")                if(P): Sheet.write (I,j,str (p.string). Strip ())Else: Sheet.write (I,j,str (td.string). Strip ()) J+ = 1I+ = 1Self.book.save ('Jiaowuxitong.xls')            defgetaverage (self): I=0 Sheet= Open_workbook ('Jiaowuxitong.xls', formatting_info=True) Table=sheet.sheet_by_index (0) nrows=table.nrows TMP1=0 TMP2=0 forIinchRange (nrows):if "Compulsory" inchTable.cell (i,5). VALUE:TMP2+ = Float (Table.cell (i,4). Value)if 'through' inchTable.cell (i,6). VALUE:TMP1+ = Float (Table.cell (i,4). Value) *60Else: Tmp1+ = Float (Table.cell (i,4). Value) *float (Table.cell (i,6). Value) I+ = 1Ave= tmp1/TMP2Print "Your average score is:"        PrintAvereturnAvedefGetgpa (self): I=0 Sheet= Open_workbook ('Jiaowuxitong.xls', formatting_info=True) Table=sheet.sheet_by_index (0) nrows=table.nrows TMP1=0 TMP2=0 forIinchRange (nrows):if  not "through" inchTable.cell (i,6). Value:tcredit= Float (Table.cell (i,4). Value) Tmp1+=Tcredit Tgrade= Float (Table.cell (i,6). Value)ifTgradeinchRange (90,101): Tmp2+ =TcreditelifTgradeinchRange (85,90): Tmp2+ = 3.7 *TcreditelifTgradeinchRange (82,84): Tmp2+ = 3.3 *TcreditelifTgradeinchRange (78,82): Tmp2+ = 3.0 *TcreditelifTgradeinchRange (75,78): Tmp2+ = 2.7 *TcreditelifTgradeinchRange (71,75): Tmp2+ = 2.3 *TcreditelifTgradeinchRange (66,71): Tmp2+ = 2.0 *TcreditelifTgradeinchRange (62,66): Tmp2+ = 1.7 *TcreditelifTgradeinchRange (60,62): Tmp2+ = 1.3 *TcreditElse: TMP2+=0 GPA= tmp2/TMP1Print "your points of achievement are:"         PrintGPAreturnGPA Reload (SYS) sys.setdefaultencoding ('Utf-8') SDU=SDU () Sdu.getgrade () Sdu.getaverage () Sdu.getgpa ( )

The introduction of web crawler--Case two: Crawling students ' achievements in the educational administration system