PHP web crawler, how to solve

PHP web crawler
Do you have a master who has developed a similar program? I can give you some pointers. Functional requirements are automatically obtained from the site and then stored in the database. PHP web crawler Database Industry Data

Share to:

------Solution--------------------
Curl crawls to the target site, the regular or DOM gets the appropriate data, and then saves the database or file.
There is no such thing as too difficult. What you need to consider are:
Crawl strategy (crawl only specific domain names, depth first or breadth first).

Crawl efficiency (multi-threaded to crawl, how to assign the task of each crawler)

And so on .....

