Libcurl multi-line blocks until those setting is unsafe (go)

From http://www.cnblogs.com/kex1n/p/4135263.html

(1), timeout (timeout)

Libcurl is a very good library, supporting HTTP,FTP and many other protocols. The biggest thing about using a library is that it's dangerous to just look at the document and write a program without looking at it. My program crashes, I suspect it is my own code to write the problem, and later found that the library is useless. Do not look closely at the document (sometimes the document itself is poor, at this time in addition to look carefully, but also more brain, consider how it is achieved), the consequences are very serious. It's sometimes nice to use someone's library or code without thinking about it, but when something goes wrong, it's not easy.

1. Curlcode Curl_global_init (long flags); In a multithreaded application, this function needs to be called in the main thread. This function sets the desired environment for the libcurl. Typically, if you do not explicitly call it, the first time you call Curl_easy_init (), Curl_easy_init calls Curl_global_init, which is not a problem in a single-threaded environment. But the multithreading is not, because Curl_global_init is not thread-safe. The tragedy occurs when Curl_easy_int is called in multiple threads, and then if two threads discover that Curl_global_init has not yet been called and calls Curl_global_init at the same time. The probability of this happening is very small, but the likelihood is there.

2. Libcurl has a good feature that can even control the timeout of domain name resolution. However, by default, it is implemented using alarm + siglongjmp. It is almost impossible to use alarm to do overtime in multiple threads. If you just use alarm, it doesn't cause the program to crash, but, with siglongjmp, it kills you (the program crashes horribly and the core doesn't see much useful information) because it needs a SIGJMP_BUF global variable and multithreading modifies it. (Typically, you can have a variable of type SIGJMP_BUF per thread, in which case it is no problem to use siglongjmp in multiple threads, but Libcurl has only one global variable, which all threads will use).

This is similar to the timeout setting for curl_easy_setopt (Curl, Curlopt_timeout, 30L), which results in the use of alarm (estimated to occur in the domain name resolution phase), as mentioned earlier, which is not possible in multiple threads. The workaround is to disable alarm this timeout, curl_easy_setopt (curl, curlopt_nosignal, 1L).

This makes it safe to use timeouts in multiple threads. But the domain name resolution has no time-out mechanism, encountered very slow domain name resolution, also very troublesome. The documentation recommendation is consider building Libcurl with C-ares support to enable asynchronous DNS lookups, which enables nice timeouts for n ame resolves without signals. C-ares is an asynchronous DNS solution.

Citation: http://gcoder.blogbus.com/logs/54871550.html

Call Libcurl download, and then use Netstat view to see that a large number of TCP connections remain in the close_wait state
To view the documentation description for Libcurl, there is an option:

Curlopt_forbid_reuse

Pass a long. Set to 1, the next transfer explicitly close the connection when done. Normally, Libcurl keeps all connections alive when do with one transfer in case a succeeding one follows that can re-use them. This option should is used with caution and if you are understand what it does. Set to 0 to has libcurl keep the connection open for possible later re-use (default behavior).

That is, by default Libcurl does not close immediately after a task is completed for reuse of the connection
If there is no new TCP request to reuse this connection, then only wait until the close_wait timeout, the time default is 7,200 seconds or higher, too many close_wait connections can cause performance problems

Workaround:


Curl_easy_setopt (Curl, curlopt_forbid_reuse, 1);

It's better to modify the TCP parameter to reduce the timeout of close_wait and time_wait

Libcurl multi-line blocks until those setting is unsafe (go)