Libcurl Multithreading crash problem

1, the problem source, multi-threaded use Libcurl cause the program to run a period of time after their own exit, no obvious exception. The appropriate bug could not be found.

Finally, by looking at the material and some articles on the Internet, we find that the problem of signal processing is:

curlopt_nosignal

Pass a long. If It is 1, Libcurl would not use any functions this install signal handlers or any functions this cause signals to be sent to the process. This option was mainly here to allow multi-threaded UNIX applications to still set/use all timeout options etc, without RIS King getting signals. (Added in 7.10)

If This option is set and Libcurl have been built with the standard name resolver, timeouts won't occur while the name R Esolve takes place. Consider building Libcurl with C-ares support to enable asynchronous DNS lookups, which enables nice timeouts for name Res Olves without signals.

Setting curlopt_nosignal to 1 makes Libcurl don't ask the system to ignore sigpipe signals, which otherwise is sent by the System when trying to send data to a socket which are closed in the other end. Libcurl makes an effort to never cause such sigpipes to trigger, but some operating systems has no to avoid them and Even on those this has there is some corner cases when they could still happen, contrary to our desire.

is when multiple threads are using time-out processing, while the main thread has a sleep or wait operation. If you do not set this option, Libcurl will signal to interrupt the wait and cause the program to exit.

So, set this option to 1 when you use it.

curl_setopt (Curl, curlopt_nosignal, 1L);

2, about the Libcurl library initialization and shutdown: Curl_global_init () and Curl_global_cleanup ()

These two functions are not thread-safe. Therefore, initialization and cleanup can only be done once in the main thread.

Although this is not necessarily a problem, but if you do not deal with this is still a probability of occurrence.

Question 3

Multithreaded Libcurl run for a period of time after a collapse, no definite point, no definite URL. Always look at the source code is not a problem, and finally through the debug trace discovery is to access SSL when the crash appears.

Just remembered that OpenSSL is not support multi-threaded, to do their own lock processing. In addition, the Libcurl does not support the associated lock operation.

Workaround:

Create a mutex function for OpenSSL when initializing Libcurl, a callback function passed to Openss

OpenSSL lock L function prototype: void (* func) (int, int, const char *, int)

Set by: Crypto_set_locking_callback (void (* func) (int, int, const char *, int));

Setting such a function is not enough, but also to configure a lock ID callback function, which can refer to the use of OpenSSL under the multi-threading related.

ID function prototype: unsigned int (*func) (void)

Setting mode: crypto_set_id_callback (unsigned int (*func) (void));

These two settings can be used to solve the problem of crash HTTPS multi-threaded requests.

code example:

The following is a code that references the Libcurl example

The key is the implementation of the two callback, as well as the location of the initialization lock (Init_locks) and the release Lock (Kill_locks)

#define  USE_OPENSSL               #include   <stdio.h>       #include  <pthread.h>      # include <curl/curl.h>              # define numt 4              /* we  have this global to let the callback get easy access  to it */      static pthread_mutex_t *lockarray;                #ifdef  USE_OPENSSL      # include <openssl/crypto.h>      static void lock_callback (int
 mode, int type, char *file, int line)       {         (void) file;         (void) line;         if  (mode & crypto_lock)  {           pthread_mutex_lock (& (Lockarray[type));        }         else {          pthread_mutex_ Unlock (& (Lockarray[type));        }      }               static unsigned long  thread_id (void)       {        unsigned long  ret;                ret= (unsigned  long) pthread_self ();        return (ret);       }  &NBsp;           static void init_locks (void)       {        int i;                 lockarray= (pthread_mutex_t *) Openssl_malloc (CRYPTO_ Num_locks ()  *                                                    sizeof (pthread _mutex_t);        for  (I=0; i<crypto_num_locks ();  i++)  {          pthread_mutex_init (& (Lockarray[i]), NULL);         }                &nbsP Crypto_set_id_callback ((unsigned long  (*) ()) thread_id);         Crypto_set_locking_callback ((void  (*) ()) lock_callback);      }               static void kill_locks (void)        {        int i;                 crypto_set_locking_callback (NULL);         for  (I=0; i<crypto_num_locks ();  i++)            pthread_mutex_destroy (& (Lockarray[i));                 openssl_free (Lockarray);      }        #endif                #ifdef &NBSP;USE_GNUTLS   &NBsp;    #include  <gcrypt.h>       #include  <errno.h>
              GCRY_THREAD_OPTION_PTHREAD_IMPL;              void init_locks (void)        {        gcry_control (GCRYCTL_SET_THREAD_CBS);       }               #define  kill_locks ()        #endif               /*  List of URLs to fetch.*/      const char *  const urls[]= {         "https://www.example.com/",          "https://www2.example.com/",         "https://www3.example.com/",        " https://www4.example.com/",       };              static void * Pull_one_url (void *url)       {        curl  *curl;                curl =  curl_easy_init ();        curl_easy_setopt (Curl, CURLOPT_URL,  url);        /* this example doesn ' t verify  The server ' s certificate, which means we        
   might be downloading stuff from an impostor */        curl_easy_setopt (curl, curlopt_ssl_verifypeer, 0l);         curl_easy_setopt (curl, curlopt_ssl_verifyhost, 0l);         Curl_easy_perform (Curl); /* ignores error */        curl _easy_cleanup (Curl);                return  NULL;      }               int main (INT&NBSP;ARGC,&NBSP;CHAR&NBSP;**ARGV)       {    
    pthread_t tid[NUMT];        int i;        int error;         (void) argc;  /* we don ' t use any arguments in this example */          (void) argv;                 /* must initialize libcurl before any threads are started * /        curl_global_init (curl_global_all);                 init_locks ();                 for (i=0; i< numt; i++)  {           error = pthread_create (&tid[i],                                   NULL, /* default attributes  please */                                  pull_one_url,                                    (void *) urls[i]);           if (0 != error)              fprintf (stderr,  "couldn ' t run thread number %d, errno %d\ n ",  i, error);          else             fprintf (stderr,  "thread %d, gets %s\n",  i,
 urls[i]);        }                /* now wait for all threads to  terminate */        for (i=0; i< numt; i++)  {           error = pthread_join (tid[i], null);           fprintf (stderr,  "thread %d terminated\n",  i);         }                kill_ Locks ();                return 0;   &NBSP;&NBSP;&NBSP;&NBSP;}

 #define  USE_OPENSSL                #include  <stdio.h>      # include <pthread.h>       #include  <curl/curl.h>                #define  NUMT 4               /* we have this global to let  the callback get easy access to it */       static pthread_mutex_t *lockarray;                #ifdef  USE_OPENSSL