Node. js and Server Load balancer under high concurrency

The emerging node. js has attracted a lot of developers, providing us with a platform to quickly build high-performance network applications. I also began to gradually invest in node. in the embrace of JS, I encountered some problems and some experience in the process of learning and using. I think it is necessary to write it out and share it as a summary.

As we all know, node. JS is based on the V8 engine, so it does not support multithreading (multi-threaded Module). To make full use of the server's multi-core, you must use the multi-process method. Load Balancing between processes is a key issue.

Multi-process shared listening socket

Node. JS and process-related modules include process, child_process, cluster, here, cluster is used to easily create the multi-process mode of the shared port (the cluster module allows you to easily create a network of processes that all share server ports ), this mode allows multiple processes to share a socket in the listening state, and the system assigns the accept connection to different sub-processes, which is also very simple to implement, cluster has done most of the work for you. Here is a test case:

 1 var cluster = require('cluster'); 2 var http = require('http'); 3 var numCPUs = require('os').cpus().length; 4  5 if (cluster.isMaster) { 6   // Fork workers. 7   for (var i = 0; i < numCPUs; i++) { 8     cluster.fork(); 9   }10 11   cluster.on('exit', function(worker, code, signal) {12     console.log('worker ' + worker.process.pid + ' died');13   });14 } else {15   // Workers can share any TCP connection16   // In this case its a HTTP server17   http.createServer(function(req, res) {18     res.writeHead(200);19     res.end("hello world\n");20   }).listen(8000);21 }

However, this completely dependent system load balancing has an important defect: on Linux and Solaris, as long as the accept queue of a sub-process is null (usually the created sub-process), the system will allocate multiple connections to the same sub-process, this will cause extremely unbalanced load between processes. Especially when a persistent connection is used, the new coming connection per unit time is not high, and the accept queue of the sub-process is usually empty, which will cause the connection to be allocated to the same process continuously. Therefore, this load balancing relies entirely on the degree of idleness of the accept queue, and can achieve Load Balancing only when short connections are used and concurrency is very high, however, the system load will be very high and the system will become unstable.

NginxHow is it done?

If you know about nginx, you may first think of the nginx processing method. nginx has one master and multiple worker processes. The master process listens to the port and is responsible for accept connection, the accept socket is sent to each worker process, which receives and processes data. In Linux, nginx uses socketpair to establish communication between master and worker processes, and uses APIs such as sendmsg and recvmsg to transmit commands and file descriptors. Does node. js support this scheme?

The answer is yes. The basis for this answer is that the child_process and cluster modules of node. js have a send method: child. Send (message, [sendhandle]).

The second parameter of this method is the socket we want to pass, and the node. js document also provides a test case:

 1 var normal = require('child_process').fork('child.js', ['normal']); 2 var special = require('child_process').fork('child.js', ['special']); 3 // Open up the server and send sockets to child 4 var server = require('net').createServer(); 5 server.on('connection', function (socket) { 6   // if this is a VIP 7   if (socket.remoteAddress === '74.125.127.100') { 8     special.send('socket', socket); 9     return;10   }11   // just the usual dudes12   normal.send('socket', socket);13 });14 server.listen(1337);

child.js

1 process.on('message', function(m, socket) {2   if (m === 'socket') {3     socket.end('You were handled as a ' + process.argv[2] + ' person');4   }5 });

Simple and refined! It seems to be a perfect solution. Let's make it a normal HTTP Server:

Master. js

 1 var http = require('http'), 2     numCPUs = require('os').cpus().length; 3     cp = require('child_process'),  4     net = require('net'); 5 var workers = []; 6 for (var i = 0; i < numCPUs; i++) { 7     workers.push(cp.fork('app.js', ['normal'])); 8 } 9 10 net.createServer(function(s) {11     s.pause();12     var worker = worker.shift();13     worker.send('c',s);14     workers.push(worker);15 }).listen(80);

 1 var http = require('http'), 2     cp = require('child_process'), 3     net = require('net'); 4 var server = http.createServer(function(req,res){ 5     res.writeHead(200, {"Content-Type": "text/plain",         "Connection": "close"}); 6     res.end("hello, world"); 7 }); 8 console.log("webServer started on " + process.pid); 9 process.on("message", function(msg,socket) {10     process.nextTick(function(){11         if(msg == 'c' && socket) {12             socket.readable = socket.writable = true;13             socket.resume();14             server.connections++;15             socket.server = server;16             server.emit("connection", socket);17             socket.emit("connect");18         }19     });20 });21                                         

We have created an HTTP server in the worker process, but this HTTP server does not listen or bind a port. When receiving the socket transmitted by the master, we call the server. emit ("connection", socket); To trigger the connection event of the server. It looks good. It works normally after a simple test. This solution is almost perfect. After experiencing the failure of the shared listening socket solution, we migrated the service to this architecture.

However, we encountered a problem. We found that the CPU and memory of the master process gradually increased and eventually reached 100%, or node. JS crash (assertion 'fd _ to_send> = 0' failed), which made me crazy. We turned to node in a helpless way. JS developers reported problems on GitHub (Issue #4587). Just that evening, node. js developersIndutnyThe problem is found mainly because the main process is not destroyed after the socket is sent to the sub-process, but is kept in the socketlist, which causes the socket to accumulate gradually in the main process, and eventually reached the upper limit.

IndutnyThis problem was quickly solved and the commit was submitted the next day. According to this commit,IndutnyThe third optional parameter is added to the send function. The modified send function is changed:

Child. Send (message, [socket], [{track: false, process: false}])

Our master process does not need to track the status of each socket, so we can set it to false. At this point, this problem has been solved perfectly, and we hope this improvement can be released along with the next version of node. js.

-------------------------------------------

Node. js officially released the 0.9.7 (unstable) version in January 18, which includes this improvement.