Are you hungry? There are so many unlicensed shops! Or big Python is more awesome!

Source: Internet
Author: User

First look at the crawl, there are so many shops do not have a business license.

Second, the operating environment

    • Python3

    • Pymongo

    • Requests

Third, analysis

First visit the Hungry home page, enter the area you want to search, and the page returns to the nearby store.

The page browser address is as follows:

https://www.ele.me/place/wtw39y8614v4?latitude=31.237236&longitude=121.36636

Open the Browser developer tool, analyze the data that needs to be crawled, then fetch the page address with requests, and use the BeautifulSoup to parse the data and return the empty value data.

Review the page information again, scroll down the page and discover that the XHR asynchronous load type has returned JSON-formatted data. As shown in the following:

Wheat-Mouth hu

Links: https://www.jianshu.com/p/19c38bfa7388

Source: Pinterest

Copyright belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please specify the source.

Where parameters:

1, Longitude,latitude adopted the German coordinate system corresponding to the latitude and longitude of the search area

2, limit=24 each limit to return a maximum of 24 stores; set up more than 30 back

3, offset=24 each return of the number of stores, 24 orders of magnitude increase, such as the page continues to turn offset=48,72 ...

More store data can be obtained by increasing offset offsets from the loop.

This page we need to collect name, phone, address that is the merchant name, telephone, business addresses three data can be. A page can collect 24 business data, JSON data such as:

The code is as follows:

Import Requestsimport jsonurl= ' https://www.ele.me/restapi/shopping/restaurants?extras[]=activities&geohash= Wtw39y8614v4&latitude=31.237236&limit=24&longitude=121.36636&offset=24&terminal=web ' r= Requests.get (URL). Textd=json.loads (R) for V in D:print (v[' name '],v[' phone '],v[' address ')

The returned results are as follows:

Four, the code

V. Results show

Sorted by address in MongoDB, sort ({address:1}) sorted by address ascending

Have you learned it?

You are welcome to follow my blog: ttps://home.cnblogs.com/u/python1234

Welcome to join thousands of people to answer questions group: 125240963

Are you hungry? There are so many unlicensed shops! Or big Python is more awesome!

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.