Crawl Source-PHP has no full-featured Web data collection open source project?

Is there an open source tool to collect data from Web pages?

For example, to include continuous rule fetching, such as fetching paging information, getting the detail page from the details page, fetching the actual DOM fields that are needed

Contains the last custom save to the database,

Contains the ability to forge IP, etc.

Includes automatic queue mechanism, automatic delay

Wait a minute

Thank you

Reply content:

Is there an open source tool to collect data from Web pages?

For example, to include continuous rule fetching, such as fetching paging information, getting the detail page from the details page, fetching the actual DOM fields that are needed

Contains the last custom save to the database,

Contains the ability to forge IP, etc.

Includes automatic queue mechanism, automatic delay

Wait a minute

Thank you

Yes, you can try the "God Arrow Hand cloud Crawler development platform." 】
The arrow Hand Cloud Crawler is a SaaS service platform that helps JS developers quickly develop crawler systems. God Archer provides a simple, flexible and open cloud Crawler development framework, so that developers only need to write a few lines of JS code online can implement a crawler. And the crawler will automatically run on the cloud server, crawling faster and more efficient.

phpcrawler,php Crawler, PHP collector, multi-process, multithreading

Phpquery

