Use the Snoopy class of PHP to capture images

I used the php Snoopy class for two days and found it useful. You can use fetchlinks to retrieve all the links in the request webpage. You can use fetchtext to obtain all the text information (the regular expression is used internally for processing). There are many other functions, for example, simulate a submission form.

Usage:

1. First download Snoopy class: http://sourceforge.net/projects/snoopy/
2. Instantiate an object and call the corresponding method to obtain the captured webpage information.

Example:

include 'snoopy/Snoopy.class.php';    $snoopy = new Snoopy();    $sourceURL = "http://xxxxxxxxx";$snoopy->fetchlinks($sourceURL);    $a = $snoopy->results;

It does not provide a way to get all the image addresses on the webpage. You need to obtain the image addresses in the list of all the articles on the page. Then I wrote one myself, mainly because it is important to match the regular expression.

// Match the regular expression $ reTag = "//I ";

Because of the special requirements, you only need to capture the pictures starting with "Dead htp: //" (the pictures on the external site may cause anti-leech protection and want to capture them locally first)

1. Captures the specified webpage and filters out all the expected article addresses;
2. Loop capture the article address in step 1, and then use the regular expression that matches the image to obtain the addresses of all the images on the page that comply with the rules;
3. Save the image according to the image suffix and ID (only gif and jpg are available here). If the image file exists, delete it and save it.

<Meta http-equiv = 'content-type' content = 'text/html; charset = UTF-8 '> <? Php include 'snoopy/snoopy. class. php'; $ snoopy = new Snoopy (); $ sourceURL = "http: // xxxxx"; $ snoopy-> fetchlinks ($ sourceURL ); $ a = $ snoopy-> results; $ re = "/d).html $/"; // filter requests for obtaining the specified file address foreach ($ a as $ tmp) {if (preg_match ($ re, $ tmp) {getImgURL ($ tmp) ;}} function getImgURL ($ siteName) {$ snoopy = new Snoopy (); $ snoopy-> fetch ($ siteName); $ fileContent = $ snoopy-> results; // match the regular expression of the image $ reTag = "//I"; if (preg_match ($ reTag, $ fileContent )) {$ ret = preg_match_all ($ reTag, $ fileContent, $ matchResult); for ($ I = 0, $ len = count ($ matchResult [1]); $ I <$ len; ++ $ I) {saveImgURL ($ matchResult [1] [$ I], $ matchResult [2] [$ I]);} function saveImgURL ($ name, $ suffix) {$ url = $ name. ". ". $ suffix; echo "requested image address :". $ url. "<br/>"; $ imgSavePath = "E:/xxx/style /Images/"; $ imgId = preg_replace ("/^. +/(d +) $/"," \ 1 ", $ name); if ($ suffix =" gif ") {$ imgSavePath. = "emotion";} else {$ imgSavePath. = "topic";} $ imgSavePath. = ("/". $ imgId. ". ". $ suffix); if (is_file ($ imgSavePath) {unlink ($ imgSavePath); echo "<p style = 'color: # f00; '> File ". $ imgSavePath. "already exists, will be deleted </p>" ;}$ imgFile = file_get_contents ($ url); $ flag = file_put_contents ($ imgSavePath, $ imgFile); if ($ fl Ag) {echo "<p> File". $ imgSavePath. "saved successfully </p>" ;}}?>

When using php to capture webpages: content, images, and links, I think the most important thing is regular expressions (retrieve desired data based on the captured content and specified rules ), in fact, the ideas are relatively simple, and there are not many methods used, that's just a few (and you can directly call the methods in the classes written by others to capture the content)

However, I have previously thought that php does not seem to implement the following methods. For example, if a file contains N rows (N is large), replace the contents of the rows that comply with the rules, for example, if Row 3 is aaa, it needs to be converted to bbbbb. Common Methods for modifying files:

1. Read the entire file (or read it row by row) at a time, save the final conversion result using a temporary file, and replace the original file.
2. Read data row by row, use fseek to control the position of the file pointer, and then write data to fwrite

Solution 1: When the file is large, one read operation is not feasible (Reading data row by row, writing data to a temporary file, and replacing the original file is inefficient ), solution 2: there is no problem when the length of the string to be replaced is smaller than or equal to the target value, but if it exceeds the value, it will be "out of bounds ", the data in the next row is also disrupted (it cannot be replaced with new content like the concept of "constituency" in JavaScript ).

The following code uses solution 2 for testing:

<? Php $ mode = "r +"; $ filename = "d:/file.txt"; $ fp = fopen ($ filename, $ mode); if ($ fp) {$ I = 1; while (! Feof ($ fp) {$ str = fgets ($ fp); echo $ str; if ($ I = 1) {$ len = strlen ($ str ); fseek ($ fp,-$ len, SEEK_CUR); // move the Pointer Forward to fwrite ($ fp, "123");} I ++ ;} fclose ($ fp) ;}?>

Read a row first. At this time, the file pointer actually refers to the beginning of the next line. Use fseek to move the file pointer back to the starting position of the previous line, and then use fwrite to replace it, because it is a replacement operation, without specifying the length, it will affect the data of the next row. What I want is to only operate on this row, for example, if you delete this row or replace the whole row with only one, the above example does not meet the requirements, maybe I have not found a proper method...