Use Node.js to implement a simple fastcgi server instance _node.js

This article is my recent Node.js learning process of an idea, put forward to discuss with you.

HTTP Server for Node.js

Using Node.js can be very easy to implement an HTTP service, the most simple examples such as the official website example:

Copy Code code as follows:

var http = require (' http ');
Http.createserver (function (req, res) {
Res.writehead ({' Content-type ': ' Text/plain '});
Res.end (' Hello world\n ');
}). Listen (1337, ' 127.0.0.1 ');

This quickly builds a Web service that listens for all HTTP requests on port 1337.
However, in a real production environment, we rarely use node.js directly as a user-facing front-end Web server for the following reasons:

1. Based on the Node.js single-threaded characteristics of the reasons, its robustness to the requirements of the developer is relatively high.
2. Other HTTP services may already have 80 ports on the server, rather than 80-port Web services that are obviously not user-friendly to the user.
3.node.js does not have much advantage over file IO processing, such as the need to respond to file resources such as pictures at the same time as a regular web site.
4. Distributed load scenarios are also a challenge.

Therefore, the use of Node.js as a Web service is more likely to be as a game server interface and other similar scenarios, most of which do not require direct user access and data exchange only services.

Node.js Web service based on Nginx as front-end machine

For these reasons, if the site-shaped product is built using Node.js, the usual way to do this is to place another mature HTTP server on the front end of the Node.js Web service, such as Nginx, which is most commonly used.
The Node.js Web service is then accessed using Nginx as a reverse proxy. Such as:

Copy Code code as follows:

server{
Listen 80;
server_name yekai.me;
Root/home/andy/wwwroot/yekai;

Location/{
Proxy_pass http://127.0.0.1:1337;
}

Location ~ \. (GIF|JPG|PNG|SWF|ICO|CSS|JS) $ {
root/home/andy/wwwroot/yekai/static;
}
}

This is a better solution to the above mentioned several problems.

Using FASTCGI Protocol Communication

However, there are some not very good places for the above agent approach.
One is the possibility that a scenario is required to control the immediate HTTP access of the Node.js Web service behind it. However, to solve the problem can also use their own services or rely on firewalls to block.
The other is because the agent's way is after all the network application layer of the scheme, nor is it very convenient to directly obtain and deal with the client HTTP interaction data, such as keep-alive, trunk and even cookies processing. This, of course, is also related to the ability and functional integrity of the proxy server itself.
So, I'm thinking of trying another approach, first thinking about the fastcgi approach that is now commonly used in PHP Web applications.

What is fastcgi

The Fast Universal Gateway Interface (Fast Common gateway interface/fastcgi) is a protocol that allows an interaction program to communicate with a Web server.

The fastcgi generated background is used as an alternative to CGI Web applications, and one of the most obvious features is that a fastcgi service process can be used to handle a series of requests The Web server connects environment variables and this page request through a socket such as the fastcgi process to the Web server, connecting to a usable UNIX Domain socket or a TCP/IP connection. For more background information, you can refer to the Wikipedia entry.

fastcgi realization of Node.js

So theoretically we just need to use node.js to create a fastcgi process, and then specify that the Nginx listener request is sent to this process. Since both Nginx and Node.js are based on event-driven service models, "theory" should be a solution for heaven and earth. Let's do it ourselves.
The Node.js in the net module can be used to create a socket service, in order to facilitate our choice of UNIX socket way.
The configuration of the Nginx side is slightly modified:

Copy Code code as follows:

...
Location/{
Fastcgi_pass Unix:/tmp/node_fcgi.sock;
}
...

Create a new file node_fcgi.js that reads as follows:

Copy Code code as follows:

var net = require (' net ');

var server = Net.createserver ();
Server.listen ('/tmp/node_fcgi.sock ');

Server.on (' Connection ', function (sock) {
Console.log (' connection ');

Sock.on (' Data ', function (data) {
Console.log (data);
});
});

Then run (because of the permissions, make sure that the Nginx and node scripts are running with the same user or with an account that has permission to do so, or that you will experience permission problems reading and writing sock files):

Node Node_fcgi.js

In browser access, we see that the terminal running the node script normally receives the data content, such as:

Copy Code code as follows:

Connection
< would be a 01...> to be in the same of the same (a)

This proves that our theoretical foundation has achieved the first step, and then we just need to figure out how this buffer content is resolved.

FASTCGI Protocol Foundation

The fastcgi record consists of a fixed-length prefix followed by a variable number of contents and padding bytes. The record structure is as follows:

Copy Code code as follows:

typedef struct {
unsigned char version;
unsigned char type;
unsigned char requestIdB1;
unsigned char requestIdB0;
unsigned char contentLengthB1;
unsigned char contentLengthB0;
unsigned char paddinglength;
unsigned char reserved;
unsigned char contentdata[contentlength];
unsigned char paddingdata[paddinglength];
} Fcgi_record;

VERSION:FASTCGI protocol version, now by default, it's 1.
Type: Record types, in fact, can be used as a different state, later specifically said
RequestID: Request ID, return should be corresponding, if not multiplexing concurrency, here directly with 1.
ContentLength: Content length, here the maximum length is 65535
Paddinglength: padding length, the effect is long data filled with 8 bytes of integer times, is mainly used to more effectively handle the data remain aligned, mainly performance considerations
Reserved: reserved bytes, for subsequent extensions
Contentdata: Real content data, one will specifically say
Paddingdata: Fill in the data, anyway, is 0, directly ignore the good.

Specific structure and description please refer to the official website documentation (http://www.fastcgi.com/devkit/doc/fcgi-spec.html#S3.3).

Request section

It seems to be very simple, that is, to parse the data to get the line. However, there is a hole, that is, this is defined here is the structure of data Units (records), not the entire buffer structure, the entire buffer by a record a record of such composition. At first it may not be very good to understand the students we are accustomed to front-end development, but this is the basis for understanding the FASTCGI protocol, and there are more examples to see later.
So, we need to parse a record individually and separate the records by the type we got earlier. Here is a simple function to get all the records:

Copy Code code as follows:

function Getrcds (data, CB) {
var RCDs = [],
Start = 0,
length = Data.length;
return function () {
if (start >= length) {
CB && CB (RCDS);
RCDs = null;
Return
}
var end = start + 8,
Header = Data.slice (start, end),
Version = Header[0],
Type = Header[1],
RequestID = (header[2] << 8) + header[3],
ContentLength = (Header[4] << 8) + header[5],
Paddinglength = header[6];
Start = end + ContentLength + paddinglength;

var BODY = ContentLength? Data.slice (end, contentlength): null;
Rcds.push ([Type, body, RequestID]);

return Arguments.callee ();
}
}
Use
Sock.on (' Data ', function (data) {
Getrcds (data, function (RCDs) {
})();
}

Note that this is just a simple process, if there are upload files and other complex situation this function is not suitable for the simplest demo is easy to handle first. At the same time, ignoring the RequestID parameters, if multiplexing is not ignored, and processing will need to be much more complex.
You can then process the different records according to type. The type is defined as follows:

Copy Code code as follows:

#define FCGI_BEGIN_REQUEST 1
#define FCGI_ABORT_REQUEST 2
#define FCGI_END_REQUEST 3
#define FCGI_PARAMS 4
#define Fcgi_stdin 5
#define FCGI_STDOUT 6
#define FCGI_STDERR 7
#define FCGI_DATA 8
#define Fcgi_get_values 9
#define Fcgi_get_values_result 10
#define FCGI_UNKNOWN_TYPE 11
#define FCGI_MAXTYPE (Fcgi_unknown_type)

Next can be based on the record type to parse to get the real data, below I only take the most commonly used fcgi_params, fcgi_get_values, Fcgi_get_values_result to explain, but their resolution is consistent. Other type record parsing has its own different rules, can refer to the definition of the specification implementation, I do not elaborate here.
Fcgi_params, Fcgi_get_values, Fcgi_get_values_result are "encoded name-value" type data, the standard format is: the length of the name, followed by the length of the value, followed by the name, followed by the value of the form of transmission, The length of 127 bytes or less can be encoded in one byte, while the longer length is always encoded in four bytes. The height of the first byte of the length indicates how long the length is encoded. A high of 0 means a byte encoding, and 1 means a four-byte encoding. Look at a comprehensive example, such as a long name short value:

Copy Code code as follows:

typedef struct {
unsigned char nameLengthB3; /* nameLengthB3 >> 7 = 1 * *
unsigned char nameLengthB2;
unsigned char nameLengthB1;
unsigned char nameLengthB0;
unsigned char valueLengthB0; /* valueLengthB0 >> 7 = 0 * *
unsigned char namedata[namelength
((B3 & 0x7f) <<) + (B2 <<) + (B1 << 8) + B0];
unsigned char valuedata[valuelength];
} fcgi_namevaluepair41;

corresponding to the implementation of the JS Method Example:

Copy Code code as follows:

function Parseparams (body) {
var j = 0,
params = {},
length = Body.length;
while (J < length) {
var name,
Value
Namelength,
Valuelength;
if (Body[j] >> 7 = 1) {
Namelength = ((body[j++] & 0x7f) <<) + (body[j++] <<) + (body[j++] << 8) + body[j++];
} else {
Namelength = body[j++];
}

if (Body[j] >> 7 = 1) {
Valuelength = ((body[j++] & 0x7f) <<) + (body[j++] <<) + (body[j++] << 8) + body[j++];
} else {
Valuelength = body[j++];
}

var ret = Body.asciislice (J, j + namelength + valuelength);
Name = ret.substring (0, namelength);
Value = ret.substring (namelength);
Params[name] = value;

J + + (Namelength + valuelength);
}
return params;
}

This enables a simple way to get a variety of parameters and environment variables. Perfect the previous code and show us how to get the client IP:

Copy Code code as follows:

Sock.on (' Data ', function (data) {
Getrcds (data, function (RCDs) {
for (var i = 0, L = rcds.length i < l; i++) {
var bodydata = Rcds[i],
Type = Bodydata[0],
BODY = bodydata[1];
if (body && type = = TYPES. Fcgi_params | | Type = = TYPES. Fcgi_get_values | | Type = = TYPES. Fcgi_get_values_result)) {
var params = parseparams (body);
Console.log (params. REMOTE_ADDR);
}
}
})();
}

By now we have learned the basics of the FASTCGI request section, followed by the implementation of the response section and finally a simple echo response service.

Response section

The response part is relatively simple, the simplest case only need to send two records on the line, that is fcgi_stdout and fcgi_end_request.
The content of the specific record entity is not redundant, look directly at the code:

Copy Code code as follows:

var res = (function () {
var MaxLength = Math.pow (2, 16);

function Buffer0 (len) {
return new Buffer ((New Array (len + 1)). Join (' \u0000 '));
};

function Writestdout (data) {
var rcdstdouthd = new Buffer (8),
Contendlength = Data.length,
Paddinglength = 8-contendlength% 8;

Rcdstdouthd[0] = 1;
RCDSTDOUTHD[1] = TYPES. Fcgi_stdout;
RCDSTDOUTHD[2] = 0;
RCDSTDOUTHD[3] = 1;
RCDSTDOUTHD[4] = contendlength >> 8;
RCDSTDOUTHD[5] = contendlength;
RCDSTDOUTHD[6] = paddinglength;
RCDSTDOUTHD[7] = 0;

Return Buffer.concat ([RCDSTDOUTHD, Data, Buffer0 (paddinglength)]);
};

function Writehttphead () {
Return Writestdout (New Buffer ("http/1.1 ok\r\ncontent-type:text/html; Charset=utf-8\r\nconnection:close\r\n\r\n "));
}

    function Writehttpbody (bodystr) {
        var Bodybuffer = [],
            BODY = new Buffer (BODYSTR);
        for (var i = 0, L = body.length I < L = + + MaxLength + 1) {
  & nbsp;         Bodybuffer.push (Writestdout (Body.slice (i, i + MaxLength)));
       }
        return Buffer.concat (Bodybuffer);
   }

function Writeend () {
var RCDENDHD = new Buffer (8);
Rcdendhd[0] = 1;
RCDENDHD[1] = TYPES. Fcgi_end_request;
RCDENDHD[2] = 0;
RCDENDHD[3] = 1;
RCDENDHD[4] = 0;
RCDENDHD[5] = 8;
RCDENDHD[6] = 0;
RCDENDHD[7] = 0;
Return Buffer.concat ([RCDENDHD, Buffer0 (8)]);
}

return function (data) {
Return Buffer.concat ([Writehttphead (), Writehttpbody (data), Writeend ()]);
};
})();

In the simplest case, you can send a complete response. Revise our final code:

Copy Code code as follows:

var visitors = 0;
Server.on (' Connection ', function (sock) {
visitors++;
Sock.on (' Data ', function (data) {
...
var querys = Querystring.parse (params. QUERY_STRING);
var ret = res (' Welcome, ' + (Querys.name | | ' Dear friend ') + '! You are the first ' + visitors + ' user Oh ~ ');
Sock.write (ret);
ret = NULL;
Sock.end ();
...
});

Open browser access: Http://domain/?name=yekai, you can see something like "Welcome you, yekai! You are the site of the 7th user Oh ~ ".
At this point, we have successfully used Node.js to achieve a simple fastcgi service. If you need to use it as a real service, then just refine our logic against the protocol specification.

Contrast test

Finally, the question we need to consider is whether the scheme is specific and feasible. There may have been some students to see the problem, I first put the simple pressure test results:

Copy Code code as follows:

FastCGI Way:
Clients, running Sec.
speed=27678 pages/min, 63277 bytes/sec.
requests:3295 Susceed, 1318 failed.

Clients, running Sec.
speed=22131 pages/min, 63359 bytes/sec.
requests:6523 Susceed, 854 failed.

Proxy method:
Clients, running Sec.
speed=28752 pages/min, 73191 bytes/sec.
requests:3724 Susceed, 1068 failed.

Clients, running Sec.
speed=26508 pages/min, 66267 bytes/sec.
requests:6716 Susceed, 2120 failed.

Direct access to Node.js service mode:
Clients, running Sec.
speed=101154 pages/min, 264247 bytes/sec.
requests:15729 Susceed, 1130 failed.

Clients, running Sec.
speed=43791 pages/min, 115962 bytes/sec.
requests:13898 Susceed, 699 failed.

Why is the proxy way better than the fastcgi way? That is because in the proxy scheme, the backend service is run directly from the Node.js native module, and the FASTCGI scheme is implemented by ourselves using Javascrip. However, it can also be seen that there is not much difference in the efficiency of the two programmes (of course, the contrast here is simple, if the gap in real business scenarios should be greater), and if node.js support fastcgi services, then efficiency should be more excellent.

Postscript

If you are interested to continue to play the students can see my article to achieve the example source code, the two days of research under the Protocol specification, in fact, not difficult.
At the same time, back to play UWSGI, but officials said V8 is already in the preparation of direct support.
Play very shallow, if there is a mistake welcome to correct the exchange.