Solution for ip address blocking during php collection. I did not find any information on the Internet, but I am very sorry. when I was looking for it, someone mentioned the search engine crawler USERAGENT. Although I found some materials on the Internet, I did not find them. However, I did not have to worry about it. when I found them, someone mentioned the USERAGENT that uses search engine crawlers. Even though I only mentioned a little bit, I still thought of listing my solutions,
1. use Snoopy or curl to pass the USERAGENT value of the search engine crawler.
View search engine crawler USERAGENT value: http://www.jb51.net/yunying/29357.html
2. use Snoopy or curl to pass the referer value.
For example, $ snoopy-> referer = 'http: // www.google.com ';
$ Header  = "Referer: http://www.google.com /";
3. use the Snoopy or curl proxy.
For example: $ snoopy-> proxy_host = "220.127.116.11 ";
$ Snoopy-& gt; proxy_port = "3128 ";
4. use Snoopy or curl to prevent IP addresses.
For example, $ snoopy-> rawheaders ['X _ FORWARDED_FOR '] = '2017. 0.0.1 ';
5. use php and a new route program to obtain a new IP address.
6. if you find that the re-routing is still blocked, it is possible that the other party has blocked your mac address. now all routers have forged MAC functions. you can write a program or manually modify the MAC address of the router, there are also tools for modifying the mac address of the NIC.
Bytes. Although I only mentioned a little bit, I still...