PHP Curl feature introduction and crawl 163 mailing List _php Tutorial

If you see, then you need to set up your PHP tutorial and open the library. If you are under the Windows platform, then very simple, you need to change the settings of your php.ini file, find Php_curl.dll, and cancel the previous semicolon comment on the line. As shown below:
Cancel the comment under the
Extension=php_curl.dll

If you are under Linux, then Google ranking you need to recompile your PHP, edit, you need to open the compilation parameters--The Configure command to add the "–with-curl" parameter.
A small example
If everything is ready, here is a small routine:
Copy the code code as follows:

//Initialize a Curl object
$curl = Curl_init ();
Set the URL you need to crawl
curl_setopt ($curl, Curlopt_url, ' http://jb51.net ');
Set Header
curl_setopt ($curl, Curlopt_header, 1);
Sets the curl parameter, which requires the result to be saved to a string or output to the screen.
curl_setopt ($curl, Curlopt_returntransfer, 1);
Run Curl, request a Web page
$data = curl_exec ($curl);
Close URL Request
Curl_close ($curl);
Show the data obtained
Var_dump ($data);

How to post data
The above is the code to crawl the Web page, the following is the post data to a page. Suppose we have a URL http://www.example.com/sendsms.php that handles the form, which can accept two form fields, one is the phone number, and the other is the message content.
Copy the code code as follows:

$phonenumber = ' 13912345678 ';
$message = ' This message is generated by curl and PHP ';
$curlpost = ' pnumber= '.　UrlEncode ($phonenumber).　' &message= '.　UrlEncode ($message). ' &submit=send ';
$ch = Curl_init (); Chain link Fencing
curl_setopt ($ch, Curlopt_url, ' http://www.example.com/sendsms.php ');
curl_setopt ($ch, Curlopt_header, 1);
curl_setopt ($ch, Curlopt_returntransfer, 1);
curl_setopt ($ch, Curlopt_post, 1);
curl_setopt ($ch, Curlopt_postfields, $curlpost);
$data = Curl_exec ();
Curl_close ($ch);
?>

From the above program we can see that using Curlopt_post to set the HTTP protocol post method instead of the Get method, and then set the post data to Curlopt_postfields.
About proxy servers
The following is an example of how to use a proxy server. Please note that the highlighted code, the code is very simple, I will not have to say more.
Copy the code code as follows:

$ch = Curl_init ();
curl_setopt ($ch, Curlopt_url, ' http://www.example.com ');
curl_setopt ($ch, Curlopt_header, 1);
curl_setopt ($ch, Curlopt_returntransfer, 1);
curl_setopt ($ch, Curlopt_httpproxytunnel, 1);
curl_setopt ($ch, Curlopt_proxy, ' fakeproxy.com:1080 ');
curl_setopt ($ch, curlopt_proxyuserpwd, ' User:password ');
$data = Curl_exec ();
Curl_close ($ch);
?>

　　
About SSL and cookies
About SSL is the HTTPS tutorial protocol, gas Generator you just need to curlopt_url the connection of http://into https://on it. Of course, there is also a parameter called Curlopt_ssl_verifyhost that can be set to verify the site.
For cookies, you need to understand the following three parameters:
Curlopt_cookie, set a cookie in the face of the conversation
Curlopt_cookiejar, save a cookie when the session ends
The Curlopt_cookiefile,cookie file.
HTTP Server Authentication
Finally, let's look at the HTTP server Authentication scenario.
Copy the code code as follows:

$ch = Curl_init ();
curl_setopt ($ch, Curlopt_url, ' http://www.example.com ');
curl_setopt ($ch, Curlopt_returntransfer, 1);
curl_setopt ($ch, Curlopt_httpauth, Curlauth_basic);
curl_setopt (Curlopt_userpwd, ' [Username]:[password] ')
$data = Curl_exec ();
Curl_close ($ch);
?>

Take a look at a 163 mailbox address list code that uses curl to crawl

Curl technology is to imitate the browser's actions to achieve page crawl or form submission, through this technology can achieve a lot of functions to go.
Copy the code code as follows:

error_reporting (0);
Mailbox user name (without @163.com suffix)
$user = ' papatata_test ';
Email password
$pass = ' 000000 ';
Target Mailbox
$mail _addr = Uenucom@163.com ';
Landing
$url = ' http://reg.163.com/logins.jsp tutorial? Type=1&url=http://entry.mail.163.com/coremail/fcg/ntesdoor2? Lightweight%3d1%26verifycookie%3d1%26language%3d-1%26style%3d-1 ';
$ch = Curl_init ($url);
Create a temporary file for storing cookie information
$cookie = Tempnam ('. ', ' ~ ');
$referer _login = ' http://mail.163.com ';
The returned result is stored in a variable instead of the default direct output
curl_setopt ($ch, Curlopt_returntransfer, true);
curl_setopt ($ch, Curlopt_header, true);
curl_setopt ($ch, Curlopt_connecttimeout, 120);
curl_setopt ($ch, Curlopt_post, true);
curl_setopt ($ch, Curlopt_referer, $referer _login);
$fields _post = Array (
' Username ' = $user,
' Password ' = $pass,
' Verifycookie ' =>1,
' Style ' =>-1,
' Product ' = ' mail163 ',
' SelType ' =>-1,
' Secure ' = ' on '
);
$headers _login = Array (
' User-agent ' = ' mozilla/5.0 (Windows; u; Windows NT 5.1; zh-cn; rv:1.9) gecko/2008052906 firefox/3.0 ',
' Referer ' = ' http://www.163.com '
);
$fields _string = ";
foreach ($fields _post as $key = $value)
{
$fields _string. = $key. '=' . $value. ' & ';
}
$fields _string = RTrim ($fields _string, ' & ');
curl_setopt ($ch, curlopt_cookiesession, true);
When the connection is closed, the cookie returned by the server is saved in the following file
curl_setopt ($ch, Curlopt_cookiejar, $cookie);
curl_setopt ($ch, Curlopt_httpheader, $headers _login);
curl_setopt ($ch, Curlopt_post, Count ($fields));
curl_setopt ($ch, Curlopt_postfields, $fields _string);
$result = curl_exec ($ch);
Curl_close ($ch);
Jump
$url = ' http://entry.mail.163.com/coremail/fcg/ntesdoor2?lightweight=1&verifycookie=1&language=-1& Style=-1&username=loki_wuxi ';
$ch = Curl_init ($url);
$headers = Array (
' User-agent ' = ' mozilla/5.0 (Windows; u; Windows NT 5.1; zh-cn; rv:1.9) gecko/2008052906 firefox/3.0 '
);
curl_setopt ($ch, Curlopt_returntransfer, true);
curl_setopt ($ch, Curlopt_header, true);
curl_setopt ($ch, Curlopt_connecttimeout, 120);
curl_setopt ($ch, Curlopt_post, true);
curl_setopt ($ch, Curlopt_httpheader, $headers);
Send the previously saved cookie information to the server side
curl_setopt ($ch, Curlopt_cookiefile, $cookie);
curl_setopt ($ch, Curlopt_cookiejar, $cookie);
$result = curl_exec ($ch);
Curl_close ($ch);
Get SID
Preg_match ('/sid=[^ '].*/', $result, $location);
$sid = substr ($location [0], 4,-1);
File_put_contents ('./result.txt ', $sid);
Address Book
$url = ' http://g4a30.mail.163.com/jy3/address/addrlist.jsp?sid= '. $sid. ' &gid=all ';
$ch = Curl_init ($url);
$headers = Array (
' User-agent ' = ' mozilla/5.0 (Windows; u; Windows NT 5.1; zh-cn; rv:1.9) gecko/2008052906 firefox/3.0 '
);
curl_setopt ($ch, Curlopt_returntransfer, true);
curl_setopt ($ch, Curlopt_header, true);
curl_setopt ($ch, Curlopt_connecttimeout, 120);
curl_setopt ($ch, Curlopt_post, true);
curl_setopt ($ch, Curlopt_httpheader, $headers);
curl_setopt ($ch, Curlopt_cookiefile, $cookie);
curl_setopt ($ch, Curlopt_cookiejar, $cookie);
$result = curl_exec ($ch);
Curl_close ($ch);
File_put_contents ('./result.txt ', $result);
Unlink ($cookie);
Start crawling content
Preg_match_all ('/]*> (. *?)]*> (. *?)/I ', $result, $infos, Preg_set_order);
1: Name 2: Email
Print_r ($infos);
?>

http://www.bkjia.com/PHPjc/444781.html www.bkjia.com true http://www.bkjia.com/PHPjc/444781.html techarticle if you see, then you need to set up your PHP tutorial and open the library. If you are under the Windows platform, then very simple, you need to change the settings of your php.ini file ...

