PHP-based Curl Quick start

CURL is a tool that uses URL syntax to transfer files and data, and supports many protocols, such as HTTP, FTP, Telnet, and so on. Best of all, PHP also supports the CURL library. This article describes some of the advanced features of CURL and how to use it in PHP. Why use CURL? Yes, we can get the Web content by other means. Most of the time, I'm just using simple PHP functions because I want to be lazy: The following is the reference: $content = file_get_contents ("http://www.nettuts.com");//Or$lines = File ("http ://www.nettuts.com ");//Orreadfile (http://www.nettuts.com); however, this approach lacks flexibility and effective error handling. And you can't use it to do some difficult tasks-such as handling coockies, validating, form submissions, file uploads, and more. Reference: CURL is a powerful library that supports many different protocols, options, and provides a variety of details about URL requests. Basic structure before learning more complex functions, take a look at the basic steps of creating a curl request in PHP: Initialize the Set variable execution and get the result release curl handle the following is the referenced content://1. Initialize $ch = Curl_init ();//2. Set options, including Urlcurl_setopt ($ch, Curlopt_url, "http://www.nettuts.com"), curl_setopt ($ch, Curlopt_returntransfer, 1); Curl _setopt ($ch, Curlopt_header, 0);//3. Executes and obtains the contents of the HTML document $output = curl_exec ($ch);//4. Release the curl handle Curl_close ($ch); the second step (that is, curl_setopt ()) is the most important, and all the mysticism is here. There is a long list of curl parameters that can be set to specify the details of the URL request. It may be difficult to read them all at once and to understand them, so today we'll just try some of the more commonly used and more useful options. Check for errors you can add a section that checks for errors (although this is not required): The following is the quoted content://... $output = curl_exec ($ch); if ($output = = = FALSE) {echo "Curl Error:". CurL_error ($ch);} // ... Please note that we use "= = = False" instead of "= = false" when comparing. Because we have to distinguish between the null output and the Boolean value false, the latter is the real error. Get information This is another optional setup item that can get information about this request after Curl executes: The following is the referenced content://... curl_exec ($ch); $info = Curl_getinfo ($ch); Echo ' get '. $info [' url ']. ' Time consuming '. $info [' Total_time ']. ' seconds ';//... The returned array includes the following information: "url"//Resource Network Address "Content_Type"//Content encoding "Http_code"//http Status Code "header_size"//header size "reques     T_size "//Requested Size" FILETIME "//File creation time" Ssl_verify_result "//ssl validation Result" redirect_count "//Jump Technology" total_time "//Total time-consuming "Namelookup_time"//dns query time-consuming "connect_time"//Waiting for connection time "pretransfer_time"//Pre-transmission preparation time "size_upload"//Upload data Size " Size_download "//Download Data size" speed_download "//download Speed" speed_upload "//upload Speed" download_content_length "//download content Length" UPL Oad_content_length "//upload content Length" starttransfer_time "//Start transfer Time" Redirect_time "//redirect duration browser-based redirection in the first example, we will provide a section for detecting the server Whether there is code for browser-based redirection. For example, some websites redirect Web pages based on whether they are mobile browsers or even which country the user is from. We use the Curlopt_httpheader option to set the HTTP request header information (HTTP headers) that we send out, including the user agent information and the default language. And then we'll take a look at these specialWhether the site will redirect us to a different URL. The following is the referenced content://Test with url$urls = Array ("http://www.cnn.com", "http://www.mozilla.com", "http://www.facebook.com"); Trial Browser Information $browsers = Array ("standard" = = Array ("user_agent" = "mozilla/5.0" (Windows; U Windows NT 6.1; En-us; rv:1.9.1.6) gecko/20091201 firefox/3.5.6 (. NET CLR 3.5.30729) "," language "=" en-us,en;q=0.5 ")," iph One "= = Array (" user_agent "=" mozilla/5.0 "(IPhone; U CPU like Mac OS X;    EN) applewebkit/420+ (khtml, like Gecko) version/3.0 mobile/1a537a safari/419.3 "," language "=" en "), "French" = Array ("user_agent" = "mozilla/4.0" (compatible; MSIE 7.0; Windows NT 5.1; GTB6 (. NET CLR 2.0.50727) "," language "=" fr,fr-fr;q=0.5 "), foreach ($urls as $url) {echo" URL: $ur    L\n ";        foreach ($browsers as $test _name = + $browser) {$ch = Curl_init ();        Set URL curl_setopt ($ch, Curlopt_url, $url); Set the browser's specific headEr curl_setopt ($ch, Curlopt_httpheader, Array ("user-agent: {$browser [' user_agent ']}",        "Accept-language: {$browser [' Language ']}");        Page content We do not need curl_setopt ($ch, curlopt_nobody, 1);        Simply return HTTP header curl_setopt ($ch, Curlopt_header, 1);        Returns the result instead of outputting it curl_setopt ($ch, Curlopt_returntransfer, 1);        $output = curl_exec ($ch);        Curl_close ($ch);        Do you have redirect HTTP header information? if (Preg_match ("!        Location: (. *)! ", $output, $matches) {echo" $test _name:redirects to $matches [1]\n];        } else {echo "$test _name:no redirection\n"; }} echo "\ n";} First, we create a set of URLs that need to be tested, and then specify a set of browser information that needs to be tested. Finally, through the loop test the various URLs and browser matching may be the case. Because we specify the Curl option, the output returned includes only the HTTP header information (stored in the $output). Using a simple regular, we check that the header message contains the word "Location:". Running this code should return the following result: Send data using the Post method when a GET request is initiated, the data can be passed to a URL through the query string. For example, when searching in Google, the search key is part of the URL's query string: http://www.google.com/search?q=nettuts In this case you may not need curL to simulate. Throw this URL to "file_get_contents ()" To get the same result. However, some HTML forms are submitted using the Post method. When this form is submitted, the data is sent over the HTTP request body, not the query string. For example, when using the CodeIgniter forum form, no matter what keyword you enter, it is always post to the following page: http://codeigniter.com/forums/do_search/You can use PHP scripts to simulate this URL request. First, create a new file that can accept and display the POST data, and we'll name it Post_output.php:print_r ($_post); Next, write a PHP script to perform the Curl request: The following is the referenced content: $url = "/http" Localhost/post_output.php "; $post _data = Array (" foo "=" bar "," query "=" Nettuts "," action "=" = "Submi T "); $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, $url); curl_setopt ($ch, Curlopt_returntransfer, 1);//We are in post data Oh! curl_setopt ($ch, Curlopt_post, 1);//Add the POST variables curl_setopt ($ch, Curlopt_postfields, $post _data); $output = Curl_exec ($ CH); Curl_close ($ch); Echo $output; After executing the code, you should get the following result: This script sends a POST request to post_output.php, the page $_post variable and returns, we use curl to capture this output. File upload and upload files are very similar to the previous post. Because all file upload forms are submitted via the Post method. First create a new page to receive the file, named Upload_output.php:print_r ($_files); Here is the script that really performs the file upload task: The following is the referenced content: $url = "Http://localhost/upload_ Output.php "; $post _data = Array (" foo "= = "Bar",//local file address to be uploaded "upload" = "@c:/wamp/www/test.zip"); $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, $ URL); curl_setopt ($ch, Curlopt_returntransfer, 1); curl_setopt ($ch, Curlopt_post, 1); curl_setopt ($ch, Curlopt_ Postfields, $post _data); $output = Curl_exec ($ch); Curl_close ($ch); echo $output; If you need to upload a file, just pass the file path like a post variable, But remember to add the @ sign in front of you. Executing this script should have the following output: Curl Batch (Multi curl) Curl also has an advanced feature, the batch handle (handle). This feature allows you to open multiple URL connections simultaneously or asynchronously. Here is the sample code from Php.net: The following is the reference://Create two curl Resources $ch1 = Curl_init (); $ch 2 = Curl_init ();//Specify the URL and the appropriate parameter curl_setopt ($ch 1, Curlopt_url, "http://lxr.php.net/"); curl_setopt ($ch 1, curlopt_header, 0); curl_setopt ($ch 2, Curlopt_url, "http:// www.php.net/"); curl_setopt ($ch 2, Curlopt_header, 0);//Create curl Batch Handle $MH = Curl_multi_init ();//Add the preceding two resource handle curl_multi_ Add_handle ($MH, $ch 1); Curl_multi_add_handle ($MH, $ch 2);//predefined a state variable $active = null;//Execute batch do {$MRC = Curl_multi_exec ($m h, $active);} while ($MRC = = Curlm_call_multi_perform); while ($active && $MRC = = Curlm_ok{if (Curl_multi_select ($MH)! =-1) {do {$MRC = Curl_multi_exec ($MH, $active);    } while ($MRC = = Curlm_call_multi_perform); }}//Close each handle curl_multi_remove_handle ($MH, $ch 1); Curl_multi_remove_handle ($MH, $ch 2); Curl_multi_close ($MH); All you have to do is open multiple curl handles and assign them to a batch handle. Then you just wait for it to execute in a while loop. There are two main loops in this example. The first do-while loop repeats the invocation of Curl_multi_exec (). This function is no partition (non-blocking), but will be executed as little as possible. It returns a status value, as long as the value equals the constant Curlm_call_multi_perform, there is still some urgent work to do (for example, sending the HTTP header information for the corresponding URL). That is, we need to constantly call this function until the return value changes. The next while loop continues only if the $active variable is true. This variable is passed as the second argument to Curl_multi_exec (), which represents whether there is an active connection in the batch handle as well. Next, we call Curl_multi_select (), which is "masked" until the active connection (for example, the Accept server response) appears. Once this function executes successfully, we go to another do-while loop and continue to the next URL. Let's take a look at how to put this into practice: The WordPress Connection Checker Imagine that you have a blog with a large number of articles that contain links to external websites. After a while, the amount of these links is invalidated for such reasons. Either the harmony, or the entire site has been the Kung Fu network ... We build a script below, analyze all these links, find out which sites/pages are not open or 404, and generate a report. Please note that the following is not a really usable WordPress plugin, just a script for a standalone function, just for demonstration, thank you. OK, let's get started. First, read all these links from the database: The following is the referenced content://config$db_host = ' localhost '; $db _user = ' root '; $db _pass = '; $db _name = ' WordPress '; $excluded _domains = Array (' localhost ', ' www.mydomain.com '); $max _conne  Ctions = 10;//initializes some variables $url_list = array (), $working _urls = Array (), $dead _urls = Array (), $not _found_urls = Array (); $active = null;//connected to Mysqlif (!mysql_connect ($db _host, $db _user, $db _pass)) {die (' Could not connect: '. Mysql_error ());} if (!mysql_select_db ($db _name)) {die (' Could not select DB: '. mysql_error ());}    Find all articles containing links $q = "Select Post_content from wp_posts WHERE post_content like '%href=% ' and post_status = ' publish '    and Post_type = ' post ', $r = mysql_query ($q) or Die (Mysql_error ()), while ($d = Mysql_fetch_assoc ($r)) {//with regular match links if (Preg_match_all ("!href=\" (. *?)            \ "!", $d [' post_content '], $matches)) {foreach ($matches [1] as $url) {//Exclude some domains            $tmp = Parse_url ($url);            if (In_array ($tmp [' Host '], $excluded _domains)) {continue;     }//Store the URL       $url _list []= $url; }}//Remove Duplicate link $url_list = Array_values (Array_unique ($url _list)), if (! $url _list) {die (' No URL to check ');} We first configure the database, a series of domain names to exclude ($excluded _domains), and the maximum number of concurrent connections ($max _connections). Then, connect to the database, get the articles and the included links, and collect them into an array ($url _list). The following code is a bit complicated, so I'll explain in detail in small steps: The following is the reference://1. Batch processor $MH = Curl_multi_init ();//2. Add the urlfor ($i = 0; $i < $max _connections; $i + +) {Add_url_to_multi_handle ($MH, $url _list) required for batch processing;} 3. Initial processing does {$MRC = Curl_multi_exec ($MH, $active);} while ($MRC = = Curlm_call_multi_perform);//4.        Main Loop while ($active && $MRC = = CURLM_OK) {//5. Active connection if (Curl_multi_select ($MH)! = 1) {//6. Work        do {$MRC = Curl_multi_exec ($MH, $active);        } while ($MRC = = Curlm_call_multi_perform); 7.        Any information? if ($mhinfo = Curl_multi_info_read ($MH)) {//means that the connection ends normally//8. Get information from the curl handle $chinfo = Curl            _getinfo ($mhinfo [' handle ']); 9.            A dead chain? if (! $chinfo [' Http_cOde ']) {$dead _urls []= $chinfo [' url ']; 10.            404, huh?            } else if ($chinfo [' http_code '] = = 404) {$not _found_urls []= $chinfo [' url ']; 11.            can also use} else {$working _urls []= $chinfo [' url ']; }//12.            Remove handle Curl_multi_remove_handle ($MH, $mhinfo [' handle ']);            Curl_close ($mhinfo [' handle ']); 13. Add new URL, work if (Add_url_to_multi_handle ($MH, $url _list)) {do {$MRC = Curl_mul                Ti_exec ($MH, $active);            } while ($MRC = = Curlm_call_multi_perform); }}}}//14. Curl_multi_close ($MH), echo "==dead urls==\n", echo implode ("\ n", $dead _urls). "\ n"; echo "==404 urls==\n"; Echo implode ("\ n", $not _found_urls). "\ n"; echo "==working urls==\n"; Echo implode ("\ n", $working _urls);//15.    Add Urlfunction add_url_to_multi_handle ($MH, $url _list) {static $index = 0 to the batch processor; If the URL is left unused if ($url _list[$index]) {//new curl Handle $ch = Curl_init ();        Configuration URL curl_setopt ($ch, Curlopt_url, $url _list[$index]);        Do not want to output the returned content curl_setopt ($ch, Curlopt_returntransfer, 1);        Redirect to where we go curl_setopt ($ch, curlopt_followlocation, 1);        No content body required, can save bandwidth and time curl_setopt ($ch, curlopt_nobody, 1);        Added to the batch processor Curl_multi_add_handle ($MH, $ch);        Dial the counter, the next time you call the function can add the next URL $index + +;    return true;    } else {//no new URL required to process return false; Please explain the above code below.    The ordinal number of the list corresponds to the sequential numbers in the code comment. Create a new batch processor.    Created a multi handle. Later we will create a function add_url_to_multi_handle () that adds the URL to the batch processor. Whenever this function is called, a new URL is added to the batch processor.      At first, we added 10 URLs to the batch processor (this figure was determined by $max _connections). It is necessary to run curl_multi_exec () for initialization, as long as it returns curlm_call_multi_perform there is still work to be done.    This is done primarily to create a connection, which does not wait for the full URL response.    As long as there are active connections in the batch, the main loop will persist.    Curl_multi_select () waits until a URL query generates an active connection.    Curl's work came again, mostly to get the response data. Check various information.    When a URL request is complete, an array is returned. There is a CURL handle in the returned array. We use theirGets the appropriate information for a single curl request.    If this is a dead chain or the request times out, the HTTP status code is not returned.    If this page is not found, a 404 status code will be returned. In other cases we think this link is available (of course, you can also check the 500 error and so on ...)    ）。    Remove this curl handle from the batch because it's no longer using value, turn it off! Good, now you can add another URL to come in.    Once again, the initialization work begins ... Well, it's all dry.    Close the batch processor and generate a report. Look back at the function that adds a new URL to the batch processor. Each time this function is called, the static variable $index incremented once, so that we know how many URLs are left in the process. I ran this script over my blog (the test requires that some of the error links were intentionally added) and the results are as follows: The following is the reference:  A total of about 40 URLs, only two seconds less than. When you need to check a larger number of URLs, its worry-saving effect can be imagined! If you open 10 connections at the same time, you can get up to 10 times times faster! In addition, you can use the no-partition feature of the Curl batch to handle a large number of URL requests without blocking your web script. Other useful Curl options HTTP authentication if a URL request requires HTTP-based authentication, you can use the following code: Copy content to Clipboard code: The following is the referenced content: $url = "http://www.somesite.com/members/ "; $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, $url); curl_setopt ($ch, Curlopt_returntransfer, 1);//Send user name and password curl_ Setopt ($ch, Curlopt_userpwd, "Myusername:mypassword");//You can allow it to redirect curl_setopt ($ch, curlopt_followlocation, 1);// The following options allow CURL to send user name and password curl_setopt ($ch, Curlopt_unrestricted_auth, 1) after redirection//, $output = Curl_exec ($ch); Curl_close ($ch); FTP upload PHP from the FTP class library, but you can also use CURL: The following is the reference content://Open a file pointer $file = fopen ("/path/to/file", "R");//URL contains most of the required information, "Ftp://us Ername:[email protected]:21/path/to/new/file "; $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, $url); curl_ Setopt ($ch, Curlopt_returntransfer, 1);//Upload related options curl_setopt ($ch, curlopt_upload, 1); curl_setopt ($ch, Curlopt_infile , $fp), curl_setopt ($ch, Curlopt_infilesize, FileSize ("/path/to/file"))///whether to turn on ASCII mode (useful when uploading text files) curl_setopt ($ch, CURLOPT_FTPASCII, 1); $output = Curl_exec ($ch); Curl_close ($ch); FQ you can initiate a curl request with an agent: The following is the referenced content: $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, ' http://www.example.com '); curl_setopt ($ CH, curlopt_returntransfer, 1);//Specify Proxy address curl_setopt ($ch, Curlopt_proxy, ' 11.11.11.11:8080 ');//provide user name and password if required Curl_ Setopt ($ch, curlopt_proxyuserpwd, ' User:pass '), $output = Curl_exec ($ch), Curl_close ($ch), callback function can be in a URL request process, Let curl invoke a specified callback function. For example, start using the data as soon as the content or response is downloaded, instead of waiting for the full download to complete. The following is the referenced content: $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, ' http://net.tutsplus.com '); curl_sEtopt ($ch, Curlopt_writefunction, "progress_function"), Curl_exec ($ch), Curl_close ($ch), function progress_function (    $ch, $str) {echo $str; return strlen ($STR);} This callback function must return the length of the string, otherwise this function will not work properly. In the process of receiving a URL response, the function is called whenever a packet is received. Summary today we learned about the power of the Curl Library and the flexibility of extensibility. I hope you like it. The next time you want to launch a URL request, consider curl!

PHP-based Curl Quick start