Python crawls the specified weight of the courier price

Directory

• First, get the query interface
• II. acquisition of relevant data
• Third, write a crawler script
• Iv. viewing the results of a query

background : Now this time, express the cross fly. We want to send a courier, give the choice is also a variety of (according to the size of the courier, delivery time, the distance and price, the company's service quality, to choose different courier [Logistics] company). Often the right things to choose the right way, this is the most perfect.

requirements : Want to send a approximate package of about three kg, do not know which courier company to choose the most suitable, there is a large express delivery company information and price list of the table on the cool crooked.

ideas : To obtain the Major Express logistics company query interface, incoming mail and destination, as well as weight. Parse the returned results of the query, print the relevant information is OK.

Roll up the sleeves dry, found a major courier company Freight price query site Express Nest (stealing XI ~ ~ because I am more lazy, save a lot of things) "If there is a violation of the relevant provisions of this site, please contact me immediately"

First, get the query interface

F12 Open the Debugging tool, select the Network option, enter the relevant information in the corresponding input box and click on the results. View Request URL

II. acquisition of relevant data

View the from data in the network to analyze what the data means

Third, write a crawler script

Scripting with Python3.6

"" "@author: Zzw@time:2018/5/10 21:55@file:queryprice.py" "" Import Urllib.requestimport urllib.parseimport Jsoni Mport reimport requestsclass Queryprice (object): # Initialize: Who corresponds to The courier company ID; Weight the weight of the items to be mailed kg def __init__ (self, who, weight): self.who = Int (who) self.weight = Int (weight) sel            F.url = "http://www.kuaidiwo.cn/freight/ajaxjg.php" # Gets the server response def Get_response (self): headers = { ' User-agent ': ' mozilla/5.0 (Windows NT 6.1; WOW64) chrome/63.0.3239.26 qqbrowser/10.0.1125.400 '} data = {' Expid ': self.who, ' pro Vinceid ': 9, # Shanghai ID ' Cityid ': 155, # Pudong ID ' Toprovinceid ': 1, # Beijing ID ' Tocityid ': 41, # Chaoyang ID ' weight ': self.weight} # Use requests more concise HTML = requests.post (Self.url, DA TA, headers). Text "data = Urllib.parse.urlencode (data, encoding= ' Utf-8 ') # encode data into utf-8 req = URLLIB.R Equest. Request (Self.url, DATA, headers) # encapsulation request Information resp = Urllib.request.urlopen (req) # Send Request HTML = Resp.read (). Decode (' Utf-8 ') # will Utf-8 Decoding the result of the Unicode ' # print (HTML) return HTML # Parsing returns def get_result (self): first = ' Shanghai Pudong- BEIJING Chaoyang%dkg goods freight: '% self.weight html = self.get_response () info = json.loads (html) if info[' Dat A '] and info[' JG ']: RESULT1 = first + "%s meta \t%s"% (info[' JG '), info[' data '][0][' name ']) # print (re Sult) return RESULT1 ELSE:RESULT2 = first + "{0} yuan \ t not query to this courier company Id={1}". Format (info[' JG '), SEL f.who) return str (RESULT2) If __name__ = = ' __main__ ': prices = [] for I in range (1,): Q = Queryp            Rice (i, a) result = Q.get_result () Price = Int (Re.findall (R ' (\d+) \s meta ', result) [0]) if price! = 0: Print (Result) prices.append print (' cheapest price:%s \ n The most expensive price is:%s '% (min (prices), max (prices)))

Iv. viewing the results of a query

The query is the first 50 ID (courier company) Price Catalog table, some cheap unimaginable, some expensive outrageous. Of course, the same sentence, the right service is the right price is our most suitable choice.



Written in the end: Although the query out so many courier companies, and even some have not heard of, but these Express (logistics) Company's price is also some deviation. Here again, some courier companies of the disorderly charge of the matter, some companies in the official website even a price is not, the individual areas to see the mood and the situation charges.

Python crawls the specified weight of the courier price