Introduction to basic use of curl

Curl is the next very famous Linux download library, through this library, can be very simple to implement the download of files and other operations.
See a simple example:

1. #include <curl/curl.h>
2. #include <stdio.h>
3. #include <string.h>
5. CURL *curl;
6. Curlcode Res;
8. size_t write_data (void *ptr, size_t size, size_t nmemb, void *stream)
9. {
10. if (strlen (char *) stream) + strlen ((char *) ptr) > 999999) return 0;
11. Strcat (Stream, (char *) PTR);
12. return SIZE*NMEMB;
13. }
15. Char *down_file (char *filename)
16. {
17. static Char str[10000000];
18. strcpy (str, "");
19. Return "";
    

• curl_easy_setopt (curl, curlopt_url, filename);//Set
• curl_easy_setopt (Curl, Curlo Pt_timeout, 3);//Set timeout time
• curl_easy_setopt (Curl, curlopt_writefunction, write_data);//Set the function to write data
• curl_easy_setopt (curl, Curlopt_writedata, str);//Set Variable to write data
• res = Curl_easy_perfo RM (curl);//execute download
• str[9999999] = ' \0′;
• if (curle_ok! = Res) return null;//determine if the download was successful
• return str;
• }
• int main ()
• {
• char url[200];
• curl = Curl_easy_init ();//Initialize Curl
• Char *result;
• while (fgets (URL, stdin)) {
• result = down_file (URL);
• if (result) puts (result);
• else puts ("Get error!");
• printf ("\nplease Input a URL:");
• }
• curl_easy_cleanup (curl);//Release Curl resource
• return 0;
• }

Below is a reprint of curl in detail using:

Curl->libcurl's manual can be viewed

Http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTWRITEDATA

Translator: Jgood (http://blog.csdn.net/JGood )

Translator Note: This is an introductory tutorial on how to use Libcurl. The document is not translated verbatim, but according to the author's understanding of Libcurl, the reference text is written. Some examples used in the text may not be from the original text, but the author in the process of learning, write some sample programs (the author used the Libcurl version is: 7.19.6). The main purpose of this is to better illustrate the use of certain API functions of Libcurl. Many examples refer to the example code provided by Libcurl. The sample program provided in the original example completely uses the C language, and the example I provide here uses the C + + language. Because of the limited capacity, some of the understanding and use of libcurl may be wrong, welcome criticism.

Goal

This document describes the basic ways and guidelines for how to use Libcurl correctly during application development. The document uses the C language to invoke the Libcurl interface and, of course, for other languages that are close to the C language.

The documentation is intended primarily for people who use Libcurl to develop. The application whipped the document refers to the source code you write, which uses Libcurl for data transfer.

More information on Libcurl's features and interfaces can be found on the relevant homepage.

Compiling source code

There are many different ways to compile the C language code. This is compiled using the UNIX platform. Even if you are using a different operating system, you can still get a lot of useful information by reading this document.

Compile

Your compiler must know the location of the Libcurl header file. So at compile time, you want to set the header file contains the path. You can use the Curl-config tool to get information about this:

$ curl-config–cflags

Link

After compiling the source code (this is not the source of Libcurl, you write your own program code), you must also link the target file into a single executable file. You want to link to the Libcurl library, as well as other libraries that libcurl depends on, such as the OPENSLL library. Of course, some other operating system libraries may also be required. Finally you have to set up some compilation options, and of course you can use the Curl-config tool to simplify the operation:

$curl-config–libs

Whether to use SSL

Custom compiled Libcurl. Unlike other libraries, Libcurl can be customized to compile, depending on the actual need to support certain features, such as whether to support SSL transmissions, such as HTTPS and FTPS. If you decide that you need to support SSL, you must have the correct settings at compile time. You can use ' curl-config ' to determine if the Libcurl library supports SSL:

$ curl-config–feature

AUTOCONF macro

When you write a configuration script to detect Libcurl and its corresponding settings, you can use predefined macros. Document DOCS/LIBCURL/LIBCURL.M4 tells you how to use these macros.

Portable, cross-platform code

Libcurl developers spend a lot of effort to make libcurl as normal as possible on most platforms.

Global initialization

Before an application can use Libcurl, it must first initialize the Libcurl. Libcurl only needs to be initialized once. The following statements can be used for initialization:

Curl_global_init ();

Curl_global_init () receives a parameter that tells Libcurl how to initialize. The parameter curl_global_all causes Libcurl to initialize all the submodules and some default options, which is usually a good default parameter value. There are also two optional values:

Curl_global_win32

Can only be applied to Windows platforms. It tells Libcurl to initialize the Winsock library. If the Winsock library is not initialized correctly, the application will not be able to use the socket. In the application, just initialize once.

Curl_global_ssl

If Libcurl is set to support SSL at compile time, this parameter is used to initialize the appropriate SSL library. Similarly, in an application, just initialize once.

Libcurl has a default protection mechanism, and if it detects that Curl_easy_perform is not initialized by Curl_global_init when it is called, Libcurl automatically calls the global initialization function based on the current run-time environment. But it must be clear that making the system self-initializing is not a good choice.

When the application is no longer using Libcurl, the curl_global_cleanup should be called to release the associated resources.

In the program, you should avoid calling Curl_global_init and Curl_global_cleanup multiple times. They can only be called once.

Features provided by Libcurl

It is generally better to develop at runtime based on features supported by libcurl than at compile time. You can obtain specific information about the runtime by calling the struct returned by the Curl_version_info function to determine some of the features that are supported by Libcurl in the current environment. The following is the author's call to the related function in visual studio2008 to get the Libcurl version information:

Use Easy interface

First, we introduce the API functions called Easy interface in Libcurl, all of which have the same prefix: Curl_easy.

The current version of Libcurl also provides multi interface for detailed use of these interfaces, which are described in the following sections. Before using multi interface, you should first understand how to use the easy interface.

To use Easy interface, you first have to create a handle,easy handle to perform each operation. Basically, each thread should have its own easy handle for data communication (if needed). Never share the same easy handle between multiple threads. The following function is used to get an easy handle:

CURL *easy_handle = Curl_easy_init ();

You can set properties and actions (action) on easy handle. Easy handle is like a logical connection for the next data transfer.

Use the curl_easy_setopt function to set the properties and operations of easy handle, which control how Libcurl communicates with the remote host. Once you have set the appropriate properties and actions in easy handle, they will always work on the handle. That is, reusing easy Hanle makes a request to the remote host, and the previously set properties are still in effect.

Many properties of easy handle are set using a string (an array of bytes ending in a). When you set the string property by using the curl_easy_setopt function, the strings are automatically copied inside the Libcurl, so the strings can be freed (if needed) after the related properties are set.

The most basic and commonly used property of easy handle is the URL. You should provide the appropriate URL through the Curlopt_url property:

curl_easy_setopt (Easy_handle, Curlopt_url, "Http://blog.csdn.net/JGood");

Suppose you want to get the resources on the remote host represented by the URL. You need to write a program for data transfer, and you may want to save the data you receive directly instead of simply printing them in the Output window. Therefore, you must first write a callback function to hold the received data. The prototype of the callback function is as follows:

size_t write_data (void *buffer, size_t size, size_t nmemb, void *userp);

You can use the following statement to register a callback function that will be called when the data is received:

curl_easy_setopt (Easy_handle, Curlopt_writefunction, Write_data);

You can provide a custom parameter to the callback function, Libcurl does not process the parameter, but simply passes:

curl_easy_setopt (Easy_handle, Curlopt_writedata, &internal_struct);

If you do not set the callback function to easy handle through the Curlopt_writefunction property, Libcurl will provide a default callback function that simply prints the received data to standard output. You can also pass an open file pointer to the default callback function by using the Curlopt_writedata property to output the data to a file.

Here are some platform-related points of attention. On some platforms, Libcurl cannot directly manipulate files opened by the application. Therefore, if you use the default callback function and pass a file pointer to easy handle through the Curlopt_writedata property, the application may fail. If you want your program to run on any system, you have to avoid this situation.

If you use Libcurl in the form of a Win32 Dynamic connection library, you must use Curlopt_writefunction to register the callback function when setting the Curlopt_writedata property. Otherwise the program fails (the author attempts to pass an open file pointer without explicitly setting the callback function, and the program does not crash.) Maybe I'm using the wrong way. ）。

Of course, Libcurl also supports many other attributes, and in the following space you will gradually reach them. Call the following function to perform true data communication:

Success = Curl_easy_perform (Easy_handle);

Curl_easy_perfrom will connect to the remote host, execute the necessary commands, and receive the data. When the data is received, the previously set callback function is called. Libcurl may receive only 1 bytes of data at a time, and may receive several K of data, Libcurl will pass the data to the callback function as much and as promptly as possible. The callback function returns the length of the received data. If the length of the data returned by the callback function is inconsistent with the length passed to it (that is, return length! = size * nmemb), Libcurl will terminate the operation and return an error code.

When the data is passed at the end, Curl_easy_perform returns a code indicating that the operation succeeded or failed. If you need more information about the details of the communication, you can set the Curlopt_errorbuffer property so that Libcurl caches many readable error messages.

Easy handle can be reused after a data communication has been completed. It is highly recommended that you reuse an already existing easy handle. If you create another easy handle to perform other data traffic after the data transfer is complete, Libcurl internally tries to reuse the last connection that was created.

For some protocols, the download file may include many complex sub-procedures: Logging, setting the transfer mode, selecting the current folder, and finally downloading the file data. With Libcurl, you don't have to care about all this, you simply provide a url,libcurl that will give you all the rest of the work.

The following example shows how to get the source of a Web page, save it to a local file, and output the source code to the console.

2. /**
4. * @brief callback function when Libcurl received data
6. *
8. * Save the received data to a local file and display it on the console.
10. *
12. * @param [in] buffer received data
14. * @param [in] size data length
16. * @param [in] nmemb number of data slices
18. * @param [in/out] user-defined pointer
20. * @return The length of the data obtained
22. */
26. size_t process_data (void *buffer, size_t size, size_t nmemb, void *user_p)
28. {
30. File *FP = (file *) user_p;
32. size_t return_size = fwrite (buffer, size, NMEMB, FP);
34. cout << (char *) buffer << Endl;
36. return return_size;
38. }
42. int main (int argc, char **argv)
44. {
46. Initialize Libcurl
50. Curlcode Return_code;
52. Return_code = Curl_global_init (CURL_GLOBAL_WIN32);
54. if (CURLE_OK! = Return_code)
56. {
58. Cerr << "Init Libcurl failed." << Endl;
60. return-1;
62. }
66. Get Easy handle
70. CURL *easy_handle = Curl_easy_init ();
72. if (NULL = = Easy_handle)
74. {
76. Cerr << "Get a easy handle failed." << Endl;
78. Curl_global_cleanup ();
80. return-1;
82. }
86. FILE *FP = fopen ("data.html", "ab+"); //
90. Setting the easy Handle property
94. Curl_easy_setopt (Easy_handle, Curlopt_url, Http://blog.csdn.net/JGood);
98. Curl_easy_setopt (Easy_handle, curlopt_writefunction, &process_data);
100. Curl_easy_setopt (Easy_handle, Curlopt_writedata, FP);
104. Execute data request
108. Curl_easy_perform (Easy_handle);
112. Freeing resources
118. Fclose (FP);
120. Curl_easy_cleanup (Easy_handle);
122. Curl_global_cleanup ();
126. return 0;
128. }

Multithreading issues

The first basic principle is that you should never share the same libcurl handle between threads, whether it's easy handle or multi handle (described below). A thread can only use one handle at a time.

Libcurl is thread-safe, with two exceptions: signal (signals) and SSL/TLS handler. The signal is used for time-out invalidation name resolution (timing out name resolves). Libcurl relies on other libraries to support SSL/STL, so when accessing HTTPS or FTPs URLs in a multithreaded manner, these libraries should meet some of the requirements for multithreaded operations. For details, refer to:

OpenSSL: Http://www.openssl.org/docs/crypto/threads.html#DESCRIPTION

GnuTLS: http://www.gnu.org/software/gnutls/manual/html_node/Multi_002dthreaded-applications.html

NSS: Claims to be multithreaded security.

When Libcurl not working properly

There is always a reason for transmission failure. You may have incorrectly set some Libcurl properties or do not correctly understand the meaning of certain properties, or the remote host will return some content that cannot be parsed correctly.

Here's a golden rule to deal with these problems: Setting the Curlopt_verbose property to 1,libcurl will output some of the details of the communication process. If the HTTP protocol is used, the request Header/response header is also output. Set Curlopt_header to 1, and the header information will appear in the contents of the message.

Admittedly, there are still bugs in Libcurl. When you find a bug in the process of using Libcurl, you want to be able to submit it to us so that we can fix the bug. When you submit a bug, provide detailed information about the protocol information tracked through the Curlopt_verbose property, the Libcurl version, Libcurl's customer code, operating system name, version, compiler name, version, and so on.

The more you know about the protocols involved, the less likely you are to make mistakes when using Libcurl.

Uploading data to a remote site

Libcurl provides a protocol-independent way to transmit data. So uploading a file to an FTP server is similar to the way you submit a put request to an HTTP server:

1. Create an easy handle or reuse the easy handle you created earlier.

2. Set the Curlopt_url property.

3. Write the callback function. When performing an upload, Libcurl reads the data to be uploaded via a callback function. (If you want to download data from a remote server, you can save the received data through a callback.) The prototype of the callback function is as follows:

size_t function (char *bufptr, size_t size, size_t nitems, void *userp);

The bufptr pointer represents a buffer that holds the data to be uploaded, size * Nitems is the length of the buffer data, USERP is a user-defined pointer, Libcurl does not do anything with the pointer, it simply passes the pointer. You can use this pointer to pass information between your application and Libcurl.

4. Register the callback function and set a custom pointer. The syntax is as follows:

Registering callback Functions curl_easy_setopt(Easy_handle, Curlopt_readfunction, read_function); Set a custom pointer curl_easy_setopt(Easy_handle, Curlopt_readdata, &filedata);

5. Tell Libcurl that the upload operation is performed.

curl_easy_setopt(Easy_handle, Curlopt_upload, 1L);

Some protocols may not correctly determine if the upload is complete without knowing the upload file size beforehand, so it is best to use the Curlopt_infilesize_large property beforehand: Tell it the size of the file to upload:

/* In this example, File_size must is an curl_off_t variable */ curl_easy_setopt(Easy_handle, Curlopt_infilesize_large, file_size);

6. Call Curl_easy_perform.

Next, Libcurl will complete the rest of the work. In the process of uploading the file, Libcurl will constantly invoke the previously set callback function to read the uploaded data into the buffer and perform the upload.

The following example shows how to upload a file to an FTP server. The author uses the FTP service that comes with IIS, and also sets the writable permission on the FTP.

2. /**
4. * @brief callback to read data.
6. */
8. size_t read_data (void *buffer, size_t size, size_t nmemb, void *user_p)
10. {
12. Return fread (buffer, size, NMEMB, (FILE *) user_p);
14. }
18. int main (int argc, char **argv)
20. {
22. Initialize Libcurl
26. Curlcode Code;
28. Code = curl_global_init (CURL_GLOBAL_WIN32);
30. if (Code! = CURLE_OK)
32. {
34. Cerr << "Init Libcurl failed." << Endl;
36. return-1;
38. }
42. FILE *FP = fopen ("a.html", "RB");
44. if (NULL = = fp)
46. {
48. cout << "can ' t open file." << Endl;
50. Curl_global_cleanup ();
52. return-1;
54. }
58. Get File size
62. Fseek (FP, 0, 2);
64. int file_size = Ftell (FP);
66. Rewind (FP);
70. Get Easy handle
76. CURL *easy_handle = NULL;
78. Easy_handle = Curl_easy_init ();
80. if (NULL = = Easy_handle)
82. {
84. Cerr << "Get a easy handle failed." << Endl;
86. Fclose (FP);
88. Curl_global_cleanup ();
90. return-1;
92. }
96. Set Eash Handle Property
100. Curl_easy_setopt (Easy_handle, Curlopt_url, ftp://127.0.0.1/upload.html);
104. Curl_easy_setopt (Easy_handle, Curlopt_upload, 1L);
106. Curl_easy_setopt (Easy_handle, curlopt_readfunction, &read_data);
108. Curl_easy_setopt (Easy_handle, Curlopt_readdata, FP);
110. Curl_easy_setopt (Easy_handle, Curlopt_infilesize_large, file_size);
114. Perform an upload operation
118. Code = curl_easy_perform (Easy_handle);
120. if (code = = CURLE_OK)
122. {
124. cout << "upload successfully." << Endl;
126. }
130. Freeing resources
134. Fclose (FP);
136. Curl_easy_cleanup (Easy_handle);
138. Curl_global_cleanup ();
142. return 0;
144. }

About password

When a client sends a request to the server, many protocols require a user name and password. Libcurl provides several ways to set them up.

Some protocols support specifying the user name and password directly in the URL, similar to the following: Protocol://user:[email protected]/path/. Libcurl can correctly identify the user name and password in this URL and perform the appropriate action. If you provide a special character in the user name and password, you should first URL-encode it.

You can also set the user name and password by using the Curlopt_userpwd property. The parameter is a string formatted as "User:password":

curl_easy_setopt(Easy_handle, Curlopt_userpwd, "User_name:password");

(I have a vague understanding of the following paragraphs) sometimes when you access a proxy server, you may be asked to provide a user name and password for user authentication. In this case, Libcurl provides another property curlopt_proxyuserpwd:

curl_easy_setopt(Easy_handle, Curlopt_proxyuserpwd, "User_name:password");

Under the UNIX platform, the user name and password to access FTP may be saved in the $home/.netrc file. Libcurl supports obtaining the user name and password directly from this file:

curl_easy_setopt(Easy_handle, CURLOPT_NETRC, 1L);

When using SSL, you may need to provide a private key for data-safe transport and to set the private key through CURLOPT_KEYPASSWD:

curl_easy_setopt(Easy_handle, CURLOPT_KEYPASSWD, "Keypassword"); HTTP Authentication

The previous chapter describes how to set a user name and password for a URL that requires authentication in Libcurl. When using the HTTP protocol, there are many ways for clients to provide authentication information to the server. The default HTTP authentication method is "Basic", which stores the user name and password in plaintext, BASE64 encoded in the HTTP request header, and sends it to the server. Of course it's not very safe.

The current version of Libcurl supports authentication methods such as: Basic, Digest, NTLM, Negotiate, Gss-negotiate, and SPNEGO. (Translator sigh: Engage in the web for so many years, altogether do not know these HTTP authentication way, really ashamed. You can use the Curlopt_httpauth property to set specific authentication methods:

curl_easy_setopt(Easy_handle, Curlopt_httpauth, curlauth_digest);

When sending authentication information to a proxy server, you can set the authentication method through Curlopt_proxyauth:

curl_easy_setopt(Easy_handle, Curlopt_proxyauth, CURLAUTH_NTLM);

You can also set multiple authentication methods (by bitwise AND), using ' Curlauth_any ' will allow Libcurl to choose any authentication method it supports. With multiple authentication methods set by the Curlopt_httpauth or Curlopt_proxyauth property, Libcurl chooses at run time what it considers to be the best way to communicate with the server:

curl_easy_setopt(Easy_handle, Curlopt_httpauth, curlauth_digest| Curlauth_basic); // curl_easy_setopt(Easy_handle, Curlopt_httpauth, Curlauth_any); HTTP Post

This chapter describes how to use Libcurl to submit data to an HTTP server using the Post method.

Method One, which is also the simplest way to use HTML as

Introduction to basic use of curl