PHP methods for getting all links to the web page (with demo source code download) and demo source code download

PHP methods for getting all links to the web page (with demo source code download) and demo source code download

The example in this article describes how PHP obtains all the connections on the web page. We will share this with you for your reference. The details are as follows:

Function getHtml ($ url, $ charset = 'utf-8') {$ curl = curl_init (); // curl_setopt ($ curl, CURLOPT_HTTPHEADER, array ('x-FORWARDED-FOR: 192.168.168.1 ', 'client-IP: 192.168.168.1'); // IP curl_setopt ($ curl, CURLOPT_URL, $ url ); curl_setopt ($ curl, CURLOPT_REFERER, ""); // $ user_agent = isset ($ _ SERVER ['HTTP _ USER_AGENT '])? $ _ SERVER ['HTTP _ USER_AGENT ']: 'mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/100 '; curl_setopt ($ curl, CURLOPT_USERAGENT, $ user_agent); // you only need to return the HTTP header // curl_setopt ($ curl, CURLOPT_HEADER, 1 ); // The page content is not required. // curl_setopt ($ curl, CURLOPT_NOBODY, 1); // return the result instead of curl_setopt ($ curl, CURLOPT_RETURNTRANSFER, 1 ); $ html = curl_exec ($ curl); // $ info = Curl_getinfo ($ curl); // echo var_dump ($ info); if ($ html = false) {// echo "cURL Error :". curl_error ($ ch); return '';} curl_close ($ curl); if ($ charset! = 'Utf-8') {$ html = iconv ($ charset, "UTF-8", $ html);} return $ html;} header ("Content-type: text/html; charset = UTF-8 "); include ('simple _ html_dom.php '); // to open extension = php_mbstring.dll // $ url = 'HTTP: // www.baidu.com/s? Wd = kaka '; $ url = 'HTTP: // www.163.com/'{{str_html = getHtml ($ url, 'gbk'); $ html = str_get_html ($ str_html ); $ links = $ html-> find ('A'); foreach ($ links as $ link) {$ txt = trim ($ link-> plaintext ); echo $ link-> href. '['. $ txt. '] <br>';} $ html = null;

Click here to download the complete instance code.