A new solution to Python crawler

The selection of multiple libraries for Python crawlers is repeated, and always does not know which one to choose, by trying multiple nights of choice

1. Reques
2. BeautifulSoup

The above two libraries are enough crawlers to crawl the data of the reptile site. First on the code:

• The call to the library:

• Web Link Access:

Url= ' ********************** '

• Web page reques headers construction, mainly anti-reptile site camouflage, get address in the network of XHR in the request headers User-agent, as follows: (any one JSON file can be)

• The specific code itself according to CSS or HTML format to get, the regular expression after the update, the individual code is as follows:

• The output is as follows:

A new solution to Python crawler