PHP web crawler

PHP web crawler Database industry data

Do you have a master who has developed a similar program? I can give you some pointers. Functional requirements are automatically obtained from the site and then stored in the database.

Reply to discussion (solution)

Curl crawls to the target site, the regular or DOM gets the appropriate data, and then saves the database or file.
There is no such thing as too difficult. What you need to consider are:
Crawl strategy (crawl only specific domain names, depth first or breadth first).

Crawl efficiency (multi-threaded to crawl, how to assign the task of each crawler)

And so on .....

Thank you, can you recommend some references ha, I rookie, still need to improve, thank you very much.

Thank you, can you recommend some references ha, I rookie, still need to improve, thank you very much.
Open-source crawlers are many, phpdig, and so on, if not all languages, there are many such as Java version of Nutch (the predecessor of Hadoop). If it's just a simple data extraction, there's a very simple client class, Snoopy

Many!!!! of open source

Want to find PHP source code reference

Thank you, can you recommend some references ha, I rookie, still need to improve, thank you very much.
Open-source crawlers are many, phpdig, and so on, if not all languages, there are many such as Java version of Nutch (the predecessor of Hadoop). If it's just a simple data extraction, there's a very simple client class, Snoopy
Thank you, what I need is to automatically fetch the required data from the website and then deposit it into the database.

Thank you, can you recommend some references ha, I rookie, still need to improve, thank you very much.
Open-source crawlers are many, phpdig, and so on, if not all languages, there are many such as Java version of Nutch (the predecessor of Hadoop). If it's just a simple data extraction, there's a very simple client class, Snoopy
Thank you, what I need is to automatically fetch the required data from the website and then deposit it into the database.
Just crawl some of the site's data, so you don't have to worry too much about efficiency. Direct Curl fetches the target page (if allowed, the simplest file_get_contents), and then the regular expression or DOM gets the data.

It's difficult to write yourself, is there open source? Help to recommend the next OH. Thank you, sir.

For example, I want to automatically crawl the network (without a fixed URL), name "BMW" The price of the car, then save it to a database, and then I'm reading it out. Can you write a simple code example? Thank you, sir.

