PHPCookbook Reading Notes-Chapter 1 Web Automation

There are three ways to GET the content of a page with a specified url through GET: Response); method 2 $ ccurl_init (www. ex

There are three ways to GET the content of a page with a specified url through GET: file function file_get_contents () provided by PHP () cURL extends the HTTP_Request class in PEAR // method 1 $ page = file_get_contents ('HTTP: // www.example.com/robots.txt'); // method 2 $ c = curl_init ('HTTP: // www. ex

GET the page content of a specified url through GET

There are three methods to obtain the content of a URL:

1. File function file_get_contents () provided by PHP ()
2. CURL Extension
3. HTTP_Request class in PEAR

// Method 1 $ page = file_get_contents ('HTTP: // response); // method 2 $ c = curl_init ('HTTP: // www.example.com/robots.txt'); curl_setopt ($ c, CURLOPT_RETURNTRANSFER, true); $ page = curl_exec ($ c); curl_close ($ c); // method 3require_once 'HTTP/Request. php '; $ r = new HTTP_Request ('HTTP: // www.example.com/robots.txt'); $ r-> sendRequest (); $ page = $ r-> getResponseBody ();

You can use these methods to obtain the XML document. You can use http_build_query () to create a query string. You can add username @ password to the url to access the protected page, use the HTTP_Client class of cURL and PEAR to track redirection.

Get a URL through POST

Let PHP simulate sending a POST request and obtain the server's feedback.

//1$url = 'http://www.example.com/submit.php';$body = 'monkey=uncle&rhino=aunt';$options = array('method' => 'POST', 'content' => $body);$context = stream_context_create(array('http' => $options));print file_get_contents($url, false, $context);//2$url = 'http://www.example.com/submit.php';$body = 'monkey=uncle&rhino=aunt';$c = curl_init($url);curl_setopt($c, CURLOPT_POST, true);curl_setopt($c, CURLOPT_POSTFIELDS, $body);curl_setopt($c, CURLOPT_RETURNTRANSFER, true);$page = curl_exec($c);curl_close($c);//3require 'HTTP/Request.php';$url = 'http://www.example.com/submit.php';$r = new HTTP_Request($url);$r->setMethod(HTTP_REQUEST_METHOD_POST);$r->addPostData('monkey','uncle');$r->addPostData('rhino','aunt');$r->sendRequest();$page = $r->getResponseBody();

Obtain a URL through Cookie

//2$c = curl_init('http://www.example.com/needs-cookies.php');curl_setopt($c, CURLOPT_COOKIE, 'user=ellen; activity=swimming');curl_setopt($c, CURLOPT_RETURNTRANSFER, true);$page = curl_exec($c);curl_close($c);//3require 'HTTP/Request.php';$r = new HTTP_Request('http://www.example.com/needs-cookies.php');$r->addHeader('Cookie','user=ellen; activity=swimming');$r->sendRequest();$page = $r->getResponseBody();

Get a URL through the Header

By modifying the information in the header, You can forge a Referer or User-Agent Request target URL. Many anti-leech websites often determine whether to allow downloading or accessing resources by judging the information sources in the Referer. You must have some background knowledge about HTTP headers.

Mark webpage

In fact, after simple modification, this code can also be used to replace sensitive keywords in webpages. This is a useful function in tianchao.

$body = '

I like pickles and herring.

A pickle pictureI have a herringbone-patterned toaster cozy.Herring is not a real HTML element!';$words = array('pickle','herring');$patterns = array();$replacements = array();foreach ($words as $i => $word) {    $patterns[] = '/' . preg_quote($word) .'/i';    $replacements[] = "\\0";}// Split up the page into chunks delimited by a// reasonable approximation of what an HTML element// looks like.$parts = preg_split("{(<(?:\"[^\"]*\"|'[^']*'|[^'\">])*>)}",                    $body,                    -1,  // Unlimited number of chunks                    PREG_SPLIT_DELIM_CAPTURE);foreach ($parts as $i => $part) {    // Skip if this part is an HTML element    if (isset($part[0]) && ($part[0] == '<')) { continue; }    // Wrap the words with s    $parts[$i] = preg_replace($patterns, $replacements, $part);}// Reconstruct the body$body = implode('',$parts);print $body;

Extract all links on the page

It is also a very good function that can be used for collection and other programs.

Tidy extension implementation:

$doc = new DOMDocument();$opts = array('output-xml' => true,              // Prevent DOMDocument from being confused about entities              'numeric-entities' => true);$doc->loadXML(tidy_repair_file('linklist.html',$opts));$xpath = new DOMXPath($doc);// Tell $xpath about the XHTML namespace$xpath->registerNamespace('xhtml','http://www.w3.org/1999/xhtml');foreach ($xpath->query('//xhtml:a/@href') as $node) {    $link = $node->nodeValue;    print $link . "\n";

Extract links using regular expressions:

$html = file_get_contents('linklist.html');$links = pc_link_extractor($html);foreach ($links as $link) {    print $link[0] . "\n";}function pc_link_extractor($html) {    $links = array();    preg_match_all('/]*)[\"\']?[^>]*>(.*?)<\/a>/i', $html,$matches,PREG_SET_ORDER); foreach($matches as $match) { $links[] = array($match[1],$match[2]); } return $links;

Convert text to HTML

The bbcode concept is similar to this one, so I will post this

Function pc_text2html ($ s) {$ s = htmlentities ($ s); $ grafs = split ("\ n", $ s); for ($ I = 0, $ j = count ($ grafs); $ I <$ j; $ I ++) {// convert html hyperlink $ grafs [$ I] = preg_replace ('/(ht | f) tp: \/[^ \ s &] + )/', '$ 1', $ grafs [$ I]); // convert email link $ grafs [$ I] = preg_replace ('/[^ @ \ s] + @ ([-a-z0-9] + \.) + [a-z] {2,}/I ',' $ 1', $ grafs [$ I]); // start a new paragraph $ grafs [$ I] ='

'. $ Grafs [$ I].'

';  }  return implode("\n\n",$grafs);}

Convert HTML to text

There is already ready-made code to implement this feature http://www.chuggnutt.com/html2text.php

Use this function strip_tags ().