New features of the "translated" node. js v0.12--cluster mode with Round-robin load balancing

Original: Https://strongloop.com/strongblog/whats-new-in-node-js-v0-12-cluster-round-robin-load-balancing

new features of node. JS v0.12-- cluster using polling scheduling algorithm for load Balancing

November, by Ben Noordhuis

Welcome to the first of a series of blog posts authored by node's core submitter Ben Noordhuis and Bert Belder. This series may consist of 7-8 articles that cover the new features of node. js v0.12. This article is mainly about the new polling scheduling cluster algorithm.

Review node's built-in cluster mode

Recalling the past, node's bemoaned limitation is its inherent single-threaded pattern. No matter how many cores your machine has, node uses only one (and warns the user that some operations will take advantage of a thread pool.) For most programs, this is only bucket relative to the total CPU time, so doing so does not really help with processor resources.

This is why node. JS v0.8 introduced the built-in cluster module. The cluster module allows you to start a master process as a supervisor, as well as one or more work processes that actually work.

One of the goals is to make it easier for you to create a "Shang" multi-process server. Ideally, you should be able to create enough work processes with an existing single-process program without having to modify any code.

Of course, things are not that easy. However, the cluster module allows the program to have little or no shared state, or to keep the shared state in an external resource, such as a database or Web service. Converting a program to cluster mode basically requires just a few lines of code:

var cluster = require (' cluster '); var os = require (' OS '); if (cluster.ismaster)   // Spawn as many workers as there is CPUs in the system.   for (var i = 0, n = Os.cpus (). length; I < n; i + = 1)    cluster.fork (); Else  // Start the application.  App ();

The program also does not need to know that it is running in cluster mode.
Add your app () to the following:

var http = require (' http 'function  app () {  var server = Http.createserver (function(req, res) {    res.end (' OK ');  });  Server.listen (8080, ' www.example.com ');}

The magic of the cluster module ensures that the worker process can bind the requested address and port, even if other worker processes are already listening. In addition, it ensures that the external connection is evenly distributed to the listener's working process-at least theoretically.

In node. js v0.8 and v0.10, the algorithm for assigning an external connection is straightforward. When the worker process calls HTTP. Server#listen () or net. Server#listen (), node. JS sends a message telling the main process to create and bind a server-side socket and share it with the worker process. If a socket is already bound, the main process skips the creation and binding phase, directly sharing the existing socket to the worker process.

This means that all work processes are listening to the same socket. When a new connection enters, the operating system wakes up a worker process. The process then accepts the connection and then begins to work.

Everything is fine at the moment. The operating system collects countless indicators of the running process and should therefore be in the best position to determine the scheduling of the process.

Actual situation

It is now the part of the theory that encounters complex realities. We have come to the point that the operating system considers the "best" not always equal to the "optimal" that the program Ape thinks. We have observed that, in exceptional cases, most connections are undertaken by two or three processes-especially Linux and Solaris systems.

From the operating system perspective, this makes sense: context switching (suspending a process and then reactivating another process) is a fairly resource-intensive operation. If more than one process listens to the same socket, it is the smartest thing to wake up the most recently blocked process, because the chance to perform a context switch is minimal. (Of course, the scheduler is a complex and variable nuisance; The above explanation must be a rough summary of the actual situation, but the basic premise that some processes get biased is still valid)

Not all programs have this weird situation--in fact, most of them don't--but in the case of the problem, the load imbalance can be seen.

After confirming the root cause of the problem, we have some corresponding therapies, but none is satisfactory. It is a bit of a good idea to not listen to this socket for the moment so that other working processes have the opportunity to accept new connections, but not enough. Access to "special care" work processes the proportion of accepted connections fell from 90% to 60-70%, a bit better, but not enough. And don't worry about it. This approach has a significant impact on the ability of the program to handle many short-term connections.

Things are getting clearer, like generating random numbers, assigning connection processing is so important that we can't rely on odds. After many discussions we reached a consensus--the lifeline is simply to abandon the current scheme, and switch to another solution as a whole. This is why the node. JS v0.11.2 version switches to the polling scheduling scenario: The new request connection is accepted by the main process and a worker process is selected for processing.

The algorithm currently chosen for the work process is not esoteric. As its name suggests, polling--just the next available worker process--but the test by the core developers and users shows that it works well: The request connection is now evenly distributed. There is also a plan to turn the selection algorithm into a configurable or plugin-based approach.

If you want to fall back to the previous assignment method, you can set the Cluster.schedulingpolicy:

var cluster = require (' cluster '); // Set This before calling and other cluster functions. Cluster.schedulingpolicy = cluster. Sched_none;cluster.fork ();

or use the NODE_CLUSTER_SCHED_POLICY environment variable to configure:

$ export node_cluster_sched_policy="none"# " rr" is Round-robin$ NODE App.js    

One-time methods are:

$ env node_cluster_sched_policy="none" NODE app.js    

about Windows

MS Windows is the only platform that has the old method as the default. node. JS uses the IOCP on the Windows platform to maximize performance. While good in most cases, it makes it expensive to pass connected handle objects to other processes. It may be possible to circumvent this in libuv, but it is unclear whether this is necessary: Windows ' ports are not too affected by Linux and Solaris.

New features of the "translated" node. js v0.12--cluster mode with Round-robin load balancing