Crawling the Android marketplace

Source: Internet
Author: User

1. Download the app

When you need information about some apps, you can use a browser to obtain information from the official website.

Someone has analyzed the Protocol buffer format used by Google market and provided Java implementation.

To download the app, see: Android Market API. Note that Google may change the Protocol. Even so, it is of great reference value.

 

2. Obtain app Information

If you only need to obtain the app information, you do not need to download it. You can directly use the crawler to obtain information about the Google market. Someone has done this.

See: crawling the Android marketplace

By the way, this crawler'sCode,Androidmarketcrawler

 

3. Notes.

In step 2, the author obtained an app from the United States. Many of the crawler's search marks are in English. If you need to write a Chinese version, you need to modify the settings, or set it to the English version of the crawler.

In addition, although Google market already has around 0.5 million apps, the author only crawled more than 0.15 million. For details, see: 155,200 apps available from the US

 

4. Technologies Used

Use the python concurrent network programming library eventlet. For more information, see eventlet.

Parse the webpage. The author uses pyquery. Of course, you can also use beautiful Soup.

The obtained data can be saved as a file, sqlite3, or MongoDB.

Crawler. py: https://github.com/bionicspirit/AndroidMarketCrawler/blob/master/crawler.py

 

 

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.