"Python Network Programming" uses RSA cryptographic algorithm module to simulate login Sina Weibo

First, the basic knowledge

http://blog.csdn.net/pi9nc/article/details/9734437

Second, the analog login

Because last semester took part in a big data game, need to crawl data, so just want to write a crawler crawl Sina Weibo data.

Of course crawling data is not aimless, I need to follow the key words to crawl related Weibo.

Just like Weibo has an advanced search feature, but to get more tweets, you need to log in, so you'll need to simulate a login.

The following code is modeled by the RSA Cryptographic algorithm module. It should be noted that Sina has anti-crawler, so when we log in to disguise as a browser.

The code is not written by itself, so the article type is labeled reproduced, because the code is similar, so I do not write, some of the specific code and problem analysis, I do not repeat, because the simulation is not my focus, the next I will talk to you after the crawl and Web page parsing part. As for the login, the article at the beginning of the link has a detailed tutorial, interested can see.

[Python]View PlainCopy

2. #! /usr/bin/env python
4. #coding =utf8
8. Import Urllib
10. Import Urllib2
12. Import Cookielib
14. Import Base64
16. Import re
18. Import JSON
20. Import Hashlib
22. Import RSA
24. Import Binascii
28. CJ = Cookielib. Lwpcookiejar ()
30. Cookie_support = Urllib2. Httpcookieprocessor (CJ)
32. Opener = Urllib2.build_opener (Cookie_support, Urllib2. HttpHandler)
34. Urllib2.install_opener (opener)
36. PostData = {
38. ' entry ': ' Weibo ',
40. ' Gateway ': ' 1 ',
42. ' from ': ' ,
44. ' savestate ': ' 7 ',
46. ' userticket ': ' 1 ',
48. ' ssosimplelogin ': ' 1 ',
50. ' vsnf ': ' 1 ',
52. ' vsnval ': ' ,
54. ' su ': ' ,
56. ' service ': ' Miniblog ',
58. ' servertime ': ' ,
60. ' nonce ': ' ,
62. ' pwencode ': ' rsa2 ', #加密算法
64. ' SP ': ' ,
66. ' encoding ': ' UTF-8 ',
68. ' prelt ': ' 401 ',
70. ' rsakv ': ' ,
72. ' url ': ' http://weibo.com/ajaxlogin.php?framelogin=1&callback= Parent.sinaSSOController.feedBackUrlCallBack ',
74. ' returntype ': ' META '
76. }
80. Class Weibologin:
82. def __init__ (self, username, password):
84. Self.username = Username
86. Self.password = password
90. def __get_spwd (self):
92. Rsapublickey = Int (self.pubkey, + )
94. Key = RSA. PublicKey (Rsapublickey, 65537) #创建公钥
96. Message = self.servertime + ' \ t ' + self.nonce + ' \ n ' + self.password #拼接明文js加密文件中得到
98. passwd = rsa.encrypt (message, key) #加密
100. passwd = Binascii.b2a_hex (passwd) #将加密信息转换为16进制.
102. return passwd
106. def __get_suser (self):
108. Username_ = Urllib.quote (self.username)
110. Username = base64.encodestring (username_) [:-1]
112. return username
116. def __prelogin (self):
118. Prelogin_url = ' http://login.sina.com.cn/sso/prelogin.php?entry=sso&callback=  Sinassocontroller.prelogincallback&su=%s&rsakt=mod&client=ssologin.js (v1.4.4) '% self.username
120. Response = Urllib2.urlopen (Prelogin_url)
122. p = re.compile (R'(.   ∗?)')
124. strURL = P.search (Response.read ()). Group (1)
126. DIC = Dict (eval (strurl)) #json格式的response
128. Self.pubkey = str (dic.get (' PubKey '))
130. self.servertime = str (dic.get (' servertime '))
132. self.nonce = str (dic.get (' nonce '))
134. self.rsakv = str (dic.get (' rsakv '))
138. def login (self):
140. url = ' http://login.sina.com.cn/sso/login.php?client=ssologin.js (v1.4.18) '
142. Try:
144. self.__prelogin () #预登录
146. except:
148. print ' prelogin Error '
150. return
152. Global PostData
154. postdata[' servertime '] = self.servertime
156. postdata[' nonce '] = self.nonce
158. postdata[' su '] = self.__get_suser ()
160. postdata[' sp '] = self.__get_spwd ()
162. postdata[' rsakv '] = self.rsakv
164. PostData = Urllib.urlencode (postdata)
166. headers = {' user-agent ':' mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:37.0) gecko/20100101 firefox/37.0 '} #伪装成浏览器
168. req = Urllib2. Request (
170. url = URL,
172. data = PostData,
174. headers = Headers
176. )
178. result = Urllib2.urlopen (req)
180. Text = Result.read ()
182. p = re.compile (' location\.replace\ '(. ∗?)  \ ")
184. Try:
186. Login_url = P.search (text). Group (1)
188. Urllib2.urlopen (Login_url)
190. print "Login succeed!"
192. except:
194. print ' Login error! '

"Python Network Programming" uses RSA cryptographic algorithm module to simulate login Sina Weibo