Solution to ip address blocking during php collection

Recently, many collection programs have been written for various websites. when I picked more than 100 websites, I suddenly found that the website of the other party could not be opened. I guess it must have been blocked by an ip address, if a proxy is used, it will still be blocked. this is not a solution. I did not find any information on the Internet, but I am very sorry. when I was looking for it, someone mentioned the search engine crawler USERAGENT. Even though I only mentioned a little bit, I still thought of listing my solutions,

1. use Snoopy or curl to pass the USERAGENT value of the search engine crawler.
View search engine crawler USERAGENT value: http://www.jb51.net/yunying/29357.html

2. use Snoopy or curl to pass the referer value.
For example, $ snoopy-> referer = 'http: // www.google.com ';
$ Header [] = "Referer: http://www.google.com /";

3. use the Snoopy or curl proxy.
For example: $ snoopy-> proxy_host = "59.108.44.41 ";
$ Snoopy-& gt; proxy_port = "3128 ";

4. use Snoopy or curl to prevent IP addresses.
For example, $ snoopy-> rawheaders ['X _ FORWARDED_FOR '] = '2017. 0.0.1 ';

5. use php and a new route program to obtain a new IP address.

6. if you find that the re-routing is still blocked, it is possible that the other party has blocked your mac address. now all routers have forged MAC functions. you can write a program or manually modify the MAC address of the router, there are also tools for modifying the mac address of the NIC.