Use Node. js to implement a simple FastCGI server instance

This article is one of my recent thoughts on Node. js learning and I will discuss it with you.

HTTP server of Node. js

Using Node. js can easily implement an http service. The simplest example is the official website example:
Copy codeThe Code is as follows:
Var http = require ('http ');
Http. createServer (function (req, res ){
Res. writeHead (200, {'content-type': 'text/plain '});
Res. end ('Hello World \ n ');
}). Listen (1337, '192. 0.0.1 ');
In this way, we quickly set up a web service that listens to all http requests on port 1337.
However, in a real production environment, we usually seldom directly use Node. js as the most front-end web server for users, mainly for the following reasons:

1. Because of Node. js's single-thread feature, its robustness ensures high requirements for developers.
2. Other http Services may already occupy port 80 on the server, but web services other than port 80 are obviously unfriendly to users.
3. Node. js does not have much advantage in file IO processing. For example, as a regular website, you may need to respond to file resources such as images at the same time.
4. Distributed Load scenarios are also a challenge.

Therefore, the use of Node. js as a web service is more likely to be used as a game server interface and other similar scenarios, most of which process services that do not require direct access by users and are only used for data exchange.

Node. js web Service Based on Nginx as the front-end server

For the above reason, if you are using Node. js-built website-like products, the conventional use is in Node. the front-end of js web Service is placed on another mature http server. For example, Nginx is the most commonly used.
Then, use Nginx as the reverse proxy to access the Node. js-based web service. For example:

Copy codeThe Code is as follows:
Server {
Listen 80;
Server_name yekai. me;
Root/home/andy/wwwroot/yekai;

Location /{
Proxy_pass http: // 127.0.0.1: 1337;
}

Location ~ \. (Gif | jpg | png | swf | ico | css | js) $ {
Root/home/andy/wwwroot/yekai/static;
}
}

In this way, the problems mentioned above are better solved.

Use FastCGI protocol for communication

However, some of the above proxy methods are not very good.
One possible scenario is to control direct http access to Node. js web Services. However, you can use your own services or block them by using the firewall.
Another reason is that the proxy method is a solution on the network application layer after all, and it is not very convenient to directly obtain and process data that interacts with the client http, for example, keep-alive, trunk, or even cookie processing. Of course, this is also related to the proxy server's own capabilities and functional perfection.
Therefore, I want to try another method. The first thing I think of is the FastCGI method that is widely used in php web applications.

What is FastCGI?

Fast Common Gateway Interface/FastCGI is a protocol that allows the interaction program to communicate with the Web server.

FastCGI is used as an alternative to cgi web applications. One of the most obvious features is that a FastCGI service process can be used to process a series of requests, the web server connects the environment variable and the page request through a socket, such as the FastCGI process, to the web server. The connection can be a Unix Domain Socket or a TCP/IP connection. For more background information, refer to the Wikipedia entry.

FastCGI Implementation of Node. js

In theory, we only need to use Node. js to create a FastCGI process, and then specify that Nginx listening requests are sent to this process. Because Nginx and Node. js are both event-driven service models, the "theory" should be a combination of Heaven and Earth solutions. Next we will implement it in person.
In Node. js, the net module can be used to establish a socket service. For convenience, we choose the unix socket method.
Slightly modify the Nginx Configuration:
Copy codeThe Code is as follows:
...
Location /{
Fastcgi_pass unix:/tmp/node_fcgi.sock;
}
...
Create a file node_fcgi.js with the following content:
Copy codeThe Code is as follows:
Var net = require ('net ');

Var server = net. createServer ();
Server. listen ('/tmp/node_fcgi.sock ');

Server. on ('connection', function (sock ){
Console. log ('connection ');

Sock. on ('data', function (data ){
Console. log (data );
});
});

Then run (for permission reasons, make sure that the Nginx and node scripts run with the same user or account with mutual permissions, or you will encounter permission issues when reading and writing sock files ):

Node node_fcgi.js

When accessing the browser, we can see that the terminal running the node script normally receives the data content, for example:
Copy codeThe Code is as follows:
Connection
<Buffer 01 01 00 01 00 08 00 00 01 00 00 00 00 00 01 01 01 01 87 01...>

This proves that our theoretical foundation has implemented the first step. Next, we only need to understand how to parse the buffer content.

FastCGI protocol Basics

A FastCGI record consists of a fixed-length prefix followed by a variable number of content and padding bytes. The record structure is as follows:
Copy codeThe Code is as follows:
Typedef struct {
Unsigned char version;
Unsigned char type;
Unsigned char requestIdB1;
Unsigned char requestIdB0;
Unsigned char contentLengthB1;
Unsigned char contentLengthB0;
Unsigned char paddingLength;
Unsigned char reserved;
Unsigned char contentData [contentLength];
Unsigned char paddingData [paddingLength];
} FCGI_Record;

Version: FastCGI Protocol version. It is good to use 1 by default.
Type: record type, which can be regarded as different states.
RequestId: Request id, which must be matched when returned. If it is not a case of multiplexing concurrency, it is better to use 1 directly.
ContentLength: content length. The maximum length here is 65535
PaddingLength: Fill length, which is an integer multiple of 8 bytes. It is mainly used to more effectively process data that maintains alignment, and mainly for performance consideration.
Reserved: reserved bytes for subsequent expansion
ContentData: real content data.
PaddingData: Fill in the data. If it is all 0, ignore it directly.

For detailed structure and instructions, please refer to the official website documentation (http://www.fastcgi.com/devkit/doc/fcgi-spec.html#S3.3 ).

Request Section

It seems very simple, that is, parsing the data once and getting it. However, there is a pitfall, that is, the Data Unit (record) structure defined here is not the entire buffer structure, and the entire buffer is composed of a record and a record. At the beginning, we may not have a good understanding of those who are used to front-end development, but this is the basis for understanding the FastCGI protocol. We will see more examples later.
Therefore, we need to parse a record separately and distinguish the Record Based on the type obtained previously. Here is a simple function for getting all records:

Copy codeThe Code is as follows:
Function getRcds (data, cb ){
Var rcds = [],
Start = 0,
Length = data. length;
Return function (){
If (start> = length ){
Cb & cb (rcds );
Rcds = null;
Return;
}
Var end = start + 8,
Header = data. slice (start, end ),
Version = header [0],
Type = header [1],
RequestId = (header [2] <8) + header [3],
ContentLength = (header [4] <8) + header [5],
PaddingLength = header [6];
Start = end + contentLength + paddingLength;

Var body = contentLength? Data. slice (end, contentLength): null;
Rcds. push ([type, body, requestId]);

Return arguments. callee ();
}
}
// Use
Sock. on ('data', function (data ){
GetRcds (data, function (rcds ){
})();
}

Note that this is only a simple process. If the function is not suitable for uploading files or other complex situations, it is easy to handle it for the simplest demonstration. At the same time, the requestId parameter is ignored. In case of multiplexing, it cannot be ignored and the processing will be much more complicated.
Next we can process different records based on the type. Type is defined as follows:
Copy codeThe Code is as follows:
# Define FCGI_BEGIN_REQUEST 1
# Define FCGI_ABORT_REQUEST 2
# Define FCGI_END_REQUEST 3
# Define FCGI_PARAMS 4
# Define FCGI_STDIN 5
# Define FCGI_STDOUT 6
# Define FCGI_STDERR 7
# Define FCGI_DATA 8
# Define FCGI_GET_VALUES 9
# Define FCGI_GET_VALUES_RESULT 10
# Define FCGI_UNKNOWN_TYPE 11
# Define FCGI_MAXTYPE (FCGI_UNKNOWN_TYPE)

Next, we can parse the actual data based on the record type. Below we will only explain the most common FCGI_PARAMS, FCGI_GET_VALUES, and FCGI_GET_VALUES_RESULT. Fortunately, their parsing methods are consistent. The parsing of other type records has its own different rules. You can refer to the standard definition implementation. I will not elaborate on it here.
FCGI_PARAMS, FCGI_GET_VALUES, and FCGI_GET_VALUES_RESULT are all data of the "encoding name-value" type. The standard format is name length, followed by value length, followed by name, followed by value, the length of 127 bytes or less can be encoded in one byte, while the longer length is always encoded in four bytes. The height of the first byte indicates the encoding method of the length. A high value of 0 indicates the encoding of one byte, and 1 indicates the encoding of four bytes. Take a comprehensive example, such as long name and short value:

Copy codeThe Code is as follows:
Typedef struct {
Unsigned char nameLengthB3;/* nameLengthB3> 7 = 1 */
Unsigned char nameLengthB2;
Unsigned char nameLengthB1;
Unsigned char nameLengthB0;
Unsigned char valueLengthB0;/* valueLengthB0> 7 = 0 */
Unsigned char nameData [nameLength
(B3 & 0x7f) <24) + (B2 <16) + (B1 <8) + B0];
Unsigned char valueData [valueLength];
} FCGI_NameValuePair41;

Example of corresponding js implementation method:

Copy codeThe Code is as follows:
Function parseParams (body ){
Var j = 0,
Params = {},
Length = body. length;
While (j <length ){
Var name,
Value,
NameLength,
ValueLength;
If (body [j]> 7 = 1 ){
NameLength = (body [j ++] & 0x7f) <24) + (body [j ++] <16) + (body [j ++] <8) + body [j ++];
} Else {
NameLength = body [j ++];
}

If (body [j]> 7 = 1 ){
ValueLength = (body [j ++] & 0x7f) <24) + (body [j ++] <16) + (body [j ++] <8) + body [j ++];
} Else {
ValueLength = body [j ++];
}

Var ret = body. asciiSlice (j, j + nameLength + valueLength );
Name = ret. substring (0, nameLength );
Value = ret. substring (nameLength );
Params [name] = value;

J + = (nameLength + valueLength );
}
Return params;
}

In this way, a simple method is implemented to obtain various parameters and environment variables. The code above demonstrates how to obtain the Client ip Address:
Copy codeThe Code is as follows:
Sock. on ('data', function (data ){
GetRcds (data, function (rcds ){
For (var I = 0, l = rcds. length; I <l; I ++ ){
Var bodyData = rcds [I],
Type = bodyData [0],
Body = bodyData [1];
If (body & (type = TYPES. FCGI_PARAMS | type = TYPES. FCGI_GET_VALUES | type = TYPES. FCGI_GET_VALUES_RESULT )){
Var params = parseParams (body );
Console. log (params. REMOTE_ADDR );
}
}
})();
}

Now we have understood the basis of the FastCGI request. Next we will implement the Response Section and finally complete a simple echo response service.

Response Section

The response part is relatively simple. In the simplest case, you only need to send two records, FCGI_STDOUT and FCGI_END_REQUEST.
The content of the object is not retained. Check the Code directly:

Copy codeThe Code is as follows:
Var res = (function (){
Var MaxLength = Math. pow (2, 16 );

Function buffer0 (len ){
Return new Buffer (new Array (len + 1). join ('\ u000000 '));
};

Function writeStdout (data ){
Var rcdStdoutHd = new Buffer (8 ),
ContendLength = data. length,
PaddingLength = 8-contendLength % 8;

RcdStdoutHd [0] = 1;
RcdStdoutHd [1] = TYPES. FCGI_STDOUT;
RcdStdoutHd [2] = 0;
RcdStdoutHd [3] = 1;
RcdStdoutHd [4] = contendLength> 8;
RcdStdoutHd [5] = contendLength;
RcdStdoutHd [6] = paddingLength;
RcdStdoutHd [7] = 0;

Return Buffer. concat ([rcdStdoutHd, data, buffer0 (paddingLength)]);
};

Function writeHttpHead (){
Return writeStdout (new Buffer ("HTTP/1.1 200 OK \ r \ nContent-Type: text/html; charset = UTF-8 \ r \ nConnection: close \ r \ n "));
}

Function writeHttpBody (bodyStr ){
Var bodyBuffer = [],
Body = new Buffer (bodyStr );
For (var I = 0, l = body. length; I <l; I ++ = MaxLength + 1 ){
BodyBuffer. push (writeStdout (body. slice (I, I + MaxLength )));
}
Return Buffer. concat (bodyBuffer );
}

Function writeEnd (){
Var rcdEndHd = new Buffer (8 );
RcdEndHd [0] = 1;
RcdEndHd [1] = TYPES. FCGI_END_REQUEST;
RcdEndHd [2] = 0;
RcdEndHd [3] = 1;
RcdEndHd [4] = 0;
RcdEndHd [5] = 8;
RcdEndHd [6] = 0;
RcdEndHd [7] = 0;
Return Buffer. concat ([rcdEndHd, buffer0 (8)]);
}

Return function (data ){
Return Buffer. concat ([writeHttpHead (), writeHttpBody (data), writeEnd ()]);
};
})();

In the simplest case, you can send a complete response. Modify the final code:
Copy codeThe Code is as follows:
Var visitors = 0;
Server. on ('connection', function (sock ){
Visitors ++;
Sock. on ('data', function (data ){
...
Var querys = querystring. parse (params. QUERY_STRING );
Var ret = res ('Welcome, '+ (querys. name | 'Dear friend') + '! You are the 1st user on this site, '+ visitors + ~ ');
Sock. write (ret );
Ret = null;
Sock. end ();
...
});
Open a browser to access: http: // domain /? Name = yekai, you can see something like "Welcome, yekai! You are the 7th-bit user on this site ~".
So far, we have successfully implemented the simplest FastCGI service using Node. js. If you want to use the service as a real service, you only need to improve our logic against the protocol specifications.

Comparison Test

Finally, we need to consider whether this solution is feasible or not? Some people may have seen the problem. Let me put the simple stress test results first:
Copy codeThe Code is as follows:
// FastCGI mode:
500 clients, running 10 sec.
Speed = 27678 pages/min, 63277 bytes/sec.
Requests: 3295 susceed, 1318 failed.

500 clients, running 20 sec.
Speed = 22131 pages/min, 63359 bytes/sec.
Requests: 6523 susceed, 854 failed.

// Proxy method:
500 clients, running 10 sec.
Speed = 28752 pages/min, 73191 bytes/sec.
Requests: 3724 susceed, 1068 failed.

500 clients, running 20 sec.
Speed = 26508 pages/min, 66267 bytes/sec.
Requests: 6716 susceed, 2120 failed.

// Directly access the Node. js service:
500 clients, running 10 sec.
Speed = 101154 pages/min, 264247 bytes/sec.
Requests: 15729 susceed, 1130 failed.

500 clients, running 20 sec.
Speed = 43791 pages/min, 115962 bytes/sec.
Requests: 13898 susceed, 699 failed.

Why is the proxy method superior to the FastCGI method? In the proxy solution, the backend service is directly run by the Node. js native module, while the FastCGI solution is implemented by using JavaScrip. However, we can also see that there is no big gap in the efficiency of the two solutions (of course, the comparison here is only a simple case, and the gap should be even larger in real business scenarios ), and if Node. js native supports FastCGI services, so the efficiency should be better.

Postscript

If you are interested in continuing to play, you can view the source code of the example implemented in this article. It is not difficult to study the protocol specification in the past two days.
At the same time, I am going back to prepare to play with uWSGI, but the official saying That v8 is already ready for direct support.
Have a good time. If you have any mistakes, please contact me.