Example _ node. js using multi-thread programming in nodejs

This article describes how to use multi-thread programming in nodejs. in this article, nodejsaddon is used to expand nodejs multi-thread programming with cc ++, for more information about Node. js, see the previous blog post. in implementing sleep in Node. js, I will introduce the usage of Node. js addon. Today's topic is addon. continue to explore the capabilities of c/c ++ to make up for Node. js weaknesses.

I have mentioned nodejs performance issues many times. In fact, in terms of language itself, nodejs has a high performance. although it is inferior to most static languages, the gap is not big. compared with other dynamic languages, the speed advantage is very obvious. But why do we often say that nodejs cannot be used in CPU-intensive scenarios? Because of its single-threaded feature, it cannot fully utilize the CPU in CPU-intensive scenarios. There is a famous Amdahl law in computer science:

Assuming that the total workload is W, it can be divided into two parts: Ws that can only be computed serially and Wp that allows parallel computing. In parallel computing with p CPUs, the performance can be improved by speedup times. Amdahl's law describes what parallel can do and cannot do. It is an ideal situation, and the actual situation is much more complicated. For example, concurrency is likely to cause resource competition, and various locks need to be added, so that concurrency is often in the waiting state. concurrency also brings the time overhead of the operating system for thread scheduling switching, add Ws. However, when Wp is much larger than Ws in a task and multiple CPU cores are available for use, the parallel performance improvement is considerable.

Okay. go back to nodejs. We imagine a computing scenario: to calculate the number of prime numbers within 4000000. In this scenario, division operations are used for programming and are not involved in operations such as memory and objects. In theory, nodejs can be ensured to run at a relatively fast speed without lagging behind c too much for comparison.

The javascript method for finding prime numbers has been provided in this blog and copied directly:

The code is as follows:

Function zhishu_js (num ){
If (num = 1 ){
Return false;
}
If (num = 2 ){
Return true;
}
For (var I = 2; I <= Math. sqrt (num); I ++ ){
If (num % I = 0 ){
Return false;
}
}
Return true;
}

Write another C language version:

The code is as follows:

# Include

Bool zhishu (int num ){
If (num = 1 ){
Return false;
}
If (num = 2 ){
Return true;
}
For (int I = 2; I <= sqrt (num); I ++ ){
If (num % I = 0 ){
Return false;
}
}
Return true;
};

In nodejs, we use a cycle from 1 to 4000000 to retrieve the prime number. in C language, we set several threads and define count as 4000000. each thread performs the following operations: if count is greater than 0, the value of count is taken out, and whether it is a prime number is calculated. at the same time, the count is reduced by 1. Based on this idea, the javascript version is easy to write:

The code is as follows:

Var count = 0;

For (j = 1; j <4000000; j ++ ){
If (zhishu (j )){
Count ++;
}
}

The key difficulty is multi-threaded programming in C language. In the early days, c/c ++ did not consider the demand for parallel computing, so the standard library did not provide multithreading support. The implementations of different operating systems are also different. To avoid this problem, we use pthread to process the thread.

Download the latest version of pthread. Since I am not familiar with gyp, link has been dependent on lib for a long time and I haven't done it yet. Finally, my method is to directly put the pthread source code under the project directory and in binding. in gyp, put pthread. c is added to the source code list and pthread is also compiled once during project compilation. The modified binding. gyp is as follows:

The code is as follows:

{
"Targets ":[
{
"Target_name": "hello ",
"Sources": ["hello. cc", "pthreads/pthread. c"],
"Include_dirs ":[
" "Pthreads"
],
"Libraries": ["Ws2_32.lib"]
}
]
}

Of course, this method is very troublesome for me. if you only add references to lib and include directories in pthread, and there is no dependency problem, it is the best and there is no need to use my method to do this.

Next we will go to C/C ++ multi-threading and define a thread processing function:

The code is as follows:

Pthread_mutex_t lock;

Void * thread_p (void * null ){
Int num, x = 0;
Do {
Pthread_mutex_lock (& lock );
Num = count --;
Pthread_mutex_unlock (& lock );
If (num> 0 ){
If (zhishu (num) x ++;
} Else {
Break;
}
} While (true );
Std: cout <''< Pthread_exit (NULL );
Return null;
}

Between the thread and thread, the variable count is competing with each other. we need to ensure that only one thread can operate the variable count at the same time. We use pthread_mutex_t lock; to add a mutex lock. When pthread_mutex_lock (& lock); is executed, the thread checks the lock status. if the lock is locked, it waits for and rechecks to block subsequent code execution. if the lock is released, it is locked, and execute the subsequent code. Correspondingly, pthread_mutex_unlock (& lock); is the unlock status.

Since the compiler performs compilation optimization while compiling, if a statement does not specify what to do and the execution of other statements does not affect it, it will be optimized by the compiler. In the above code, I added the code for counting the number of prime numbers. If this code is not added, it is like this:

The code is as follows:

For (int j = 0; j <4000000; j ++ ){
Zhishu (j );
}

Will be skipped directly by the compiler and will not run.

The description of adding addon has already been introduced. we can receive a parameter from javascript to indicate the number of threads, and then create a specified number of threads in c to complete the quality search. Complete code:

The code is as follows:

# Include
# Include
# Include
# Include "pthreads \ pthread. h"
# Define MAX_THREAD 100
Using namespace v8;

Int count = 4000000;
Pthread_t tid [MAX_THREAD];
Pthread_mutex_t lock;

Void * thread_p (void * null ){
Int num, x = 0;
Do {
Pthread_mutex_lock (& lock );
Num = count --;
Pthread_mutex_unlock (& lock );
If (num> 0 ){
If (zhishu (num) x ++;
} Else {
Break;
}
} While (true );
Std: cout <''< Pthread_exit (NULL );
Return null;
}

NAN_METHOD (Zhishu ){
NanScope ();
Pthread_mutex_init (& lock, NULL );
Double arg0 = args [0]-> NumberValue ();
Int c = 0;
For (int j = 0; j <arg0 & j Pthread_create (& tid [j], NULL, thread_p, NULL );
}
For (int j = 0; j <arg0 & j Pthread_join (tid [j], NULL );
}
NanReturnUndefined ();
}

Void Init (HandleExports ){
Exports-> Set (NanSymbol ("zhishu"), FunctionTemplate: New (Zhishu)-> GetFunction ());
}

NODE_MODULE (hello, Init );

Phread_create can be used to create a thread. the default value is joinable. at this time, the subthread is subject to the main thread. phread_join blocks the main thread and waits for the subthread to join until the subthread exits. If the sub-thread has exited, phread_join will not do anything. Therefore, thread_join is executed for all threads, which ensures that the main thread continues to run only after all threads exit.

Complete the nodejs script:

The code is as follows:

Var zhishu_c = require ('./build/Release/hello. node'). zhishu;
Function zhishu (num ){
If (num = 1 ){
Return false;
}
If (num = 2 ){
Return true;
}
For (var I = 2; I <= Math. sqrt (num); I ++ ){
If (num % I = 0 ){
Return false;
}
}
Return true;
}

Console. time ("c ");
Zhishu_c (100 );
Console. timeEnd ("c ");

Console. time ("js ");
Var count = 0;
For (j = 1; j <4000000; j ++ ){
If (zhishu (j )){
Count ++;
}
}
Console. log (count );
Console. timeEnd ("js ");

Take a look at the test results:

In a single thread, although the speed of C/C ++ is 181% of that of nodejs, we think this score is very good in dynamic languages. The speed increase in double threads is the most obvious, because my computer is a dual-core four-thread CPU, which may already be processed using two cores. The speed of 4 threads reaches the maximum. at this time, the dual-core four-line thread can reach the limit. when the thread increases, the speed cannot be increased. In Amdahl's law above, p has reached the upper limit of 4. Adding more threads will increase the scheduling time of the operating system process and the lock time. Although the competition for CPU time can also be increased, the increase of Ws is more obvious in general, performance is reduced. If you do this experiment on an idle machine, the data will be better.

From this experiment, we can draw the conclusion that for CPU-intensive operations, the efficiency will be improved by handing over to static languages, if the computation involves many operations such as memory, string, array, and recursion (which will be verified later), the performance improvement will be even more amazing. At the same time, the rational use of multithreading can effectively improve the processing efficiency, but not the more threads, the better, according to the machine's situation reasonable configuration.

Nodejs itself is really not good at processing CPU-intensive tasks, but with the experience in this article, I think it is not impossible to overcome this obstacle.