Implementation of FASTCGI protocol and in-depth understanding in PHP

Before discussing FastCGI, we have to say that the traditional CGI works, and should probably understand the CGI 1.1 protocol

Analysis of traditional CGI working principle

After the client accesses a URL address, submits the data through get/post/put and other means, and makes a request to the WEB server via the HTTP protocol, the server-side HTTP Daemon (daemon) passes the information described in the HTTP request through the standard input stdin and environment variables (environment variable) is passed to the CGI program specified by the home page and starts the application for processing (including processing of the database), the processing results are returned to the HTTP Daemon daemon through the standard output stdout, and then by the HTTP Daemon process through H The TTP protocol is returned to the client.

This paragraph of the above understanding may be more abstract, the following we have a GET request as an example for detailed explanation.

The following code is used to implement the functions expressed in the diagram. The WEB server initiates a socket listener service and then executes the CGI program locally. Later there is a more detailed code interpretation.

Web Server Code

#include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/types.h> #include < sys/socket.h> #include <arpa/inet.h> #include <netinet/in.h> #include <string.h> #define Serv_     PORT 9003char* Str_join (char *str1, char *str2); char* html_response (Char *res, char *buf); int main (void) {int LFD, CFD;    struct sockaddr_in serv_addr,clin_addr;    Socklen_t Clin_len;    Char buf[1024],web_result[1024];    int Len;    FILE *cin;        if (LFD = socket (af_inet,sock_stream,0)) = =-1) {perror ("Create socket failed");    Exit (1);    } memset (&serv_addr, 0, sizeof (SERV_ADDR));    serv_addr.sin_family = af_inet;    SERV_ADDR.SIN_ADDR.S_ADDR = htonl (Inaddr_any);    Serv_addr.sin_port = htons (Serv_port);        if (Bind (LFD, (struct sockaddr *) &serv_addr, sizeof (serv_addr)) = =-1) {perror ("bind error");    Exit (1);        } if (listen (LFD) = =-1) {perror ("Listen error");    Exit (1); } Signal (SIGCLD, sig_ign);        while (1) {Clin_len = sizeof (CLIN_ADDR);            if (cfd = accept (LFD, struct sockaddr *) &clin_addr, &clin_len) = = =-1) {perror ("receive error \ n");        Continue        } cin = Fdopen (CFD, "R");        Setbuf (CIN, (char *) 0); Fgets (buf,1024,cin);        Read the first line of printf ("\n%s", buf);        ============================ CGI environment variable settings demo ============================//For example "get/user.cgi?id=1 http/1.1";        Char *delim = "";        Char *p;        Char *method, *filename, *query_string;        Char *query_string_pre = "query_string=";         method = Strtok (Buf,delim);             GET p = strtok (Null,delim);           /user.cgi?id=1 filename = strtok (P, "?");        /user.cgi if (strcmp (filename, "/favicon.ico") = = 0) {continue;    } query_string = Strtok (NULL, "?");        Id=1 putenv (Str_join (query_string_pre,query_string)); //============================ CGI environment variable setting demo ============================ int pid = fork ();        if (PID > 0) {close (CFD);            } else if (PID = = 0) {close (LFD);            FILE *stream = Popen (Str_join (".", filename), "R");            Fread (Buf,sizeof (char), sizeof (BUF), stream);            Html_response (WEB_RESULT,BUF);            Write (cfd,web_result,sizeof (Web_result));            Pclose (stream);            Close (CFD);        Exit (0);            } else {perror ("fork Error");        Exit (1);    }} close (LFD); return 0;}    char* Str_join (Char *str1, char *str2) {char *result = malloc (strlen (str1) +strlen (str2) +1);    if (result = = NULL) exit (1);    strcpy (result, str1);    strcat (result, str2); return result;} char* Html_response (Char *res, char *buf) {char *html_response_template = "http/1.1-ok\r\ncontent-type:text/html\r\    Ncontent-length:%d\r\nserver:mengkang\r\n\r\n%s "; sprintf (RES,HTML_RESPONse_template,strlen (BUF), buf); return res;}

As the emphasis in the above code:

• 66~81 line to find the relative path of the CGI program (we simply define its root directory as the Web program's current directory), so that we can execute the CGI program in the child process, and set the environment variables to facilitate the CGI program to read while running;

• 94~95 writes the standard output of the CGI program to the cache of the Web server daemon;

• 97 lines writes the wrapped HTML result to the client socket descriptor, which is returned to the client connecting to the Web server.

CGI Program (USER.C)

#include <stdio.h> #include <stdlib.h>//queries the user's information by obtaining an ID of int main (void) {    //========================== = = Analog database ============================    typedef struct     {        int  id;        char *username;        int age  ;    } user;    User users[] = {        {},        {            1,            "Mengkang.zhou",            +        }    };    ============================ Simulation Database ============================    char *query_string;    int id;    Query_string = getenv ("query_string");    if (query_string = = NULL)    {        printf ("No input data");    } else if (sscanf (query_string, "id=%d", &id)! = 1)    {        printf ("No input ID");    } else    {        printf ("User Information query <br> Study No.:%d<br> Name:%s<br> age :%d ", id,users[id].username,users[id].age);    }    return 0;}

Compile the CGI program above gcc user.c -o user.cgi and put it in the sibling directory of the Web program above.

The 28th line in the code, which reads the environment variables previously set in the Web server daemon from the environment variables, is the focus of our presentation.

Analysis of working principle of FastCGI

Relative to the cgi/1.1 specification, the WEB server in the local fork a child process to execute a CGI program, populate the CGI predefined environment variables, put into the system environment variables, the HTTP body content through the standard input to the child process, after processing through the standard output returned to Web server. The core of the FastCGI is to outlaw the traditional fork-and-execute approach, reducing the overhead of each launch (as explained in PHP) and processing requests in a permanent manner.

The FastCGI workflow is as follows:

The FastCGI process Manager itself initializes, starts multiple CGI interpreter processes, and waits for a connection from the Web Server.

The WEB server uses Socket communication with the FastCGI process Manager to send CGI environment variables and standard input data to the CGI interpreter process via the FastCGI protocol.

The CGI interpreter process returns the standard output and error information from the same connection to the Web Server after it finishes processing.

The CGI interpreter process then waits and processes the next connection from the Web Server.

One of the differences between FastCGI and the traditional CGI pattern is that the Web server does not directly execute the CGI program, but instead interacts with the FastCGI responder (FastCGI process Manager) through the socket, and the Web server needs to encapsulate the CGI interface data in accordance with the Fast The CGI protocol packet is sent to the FastCGI responder program. Because the FastCGI process Manager is based on socket communication, it is also distributed, and the Web server and the CGI responder server are deployed separately.

Again, FastCGI is a kind of protocol, it is built on the basis of cgi/1.1, the cgi/1.1 inside the data to be passed through the FastCGI protocol definition of the order, the format of transmission.

Preparatory work

Perhaps the above content is still very abstract, this is because the first pair of fastcgi protocol does not have a general understanding of the second, there is no actual code learning. So need to learn the contents of the FastCGI protocol in advance, not necessarily need to fully understand, can be roughly understood after reading this article and then combine learning to understand digestion.

Http://www.fastcgi.com/devkit ... (English original) Http://andylin02.iteye.com/bl ... (Chinese version)

FastCGI Protocol Analysis

The following combination of PHP FastCGI code for analysis, without special instructions The following code is from the source of PHP.

FastCGI message Types

FastCGI makes a number of types of messages that are transmitted, and its structure is defined as follows:

typedef enum _FCGI_REQUEST_TYPE {    fcgi_begin_request      =  1,/* [in]                              */    fcgi_abort_request      =  2,/* [in]  (Not supported)             */    Fcgi_end_request        =  3,/* [out]                             */    fcgi_params             =  4,/* [in]  environment Variables       */    Fcgi_stdin              =  5,/* [in]  post data                   */    fcgi_stdout             =  6,/* [out] Response                    */    Fcgi_stderr             =  7,/* [out] errors                      */    fcgi_data               =  8,/* [in]  Filter data (not supported) *    / Fcgi_get_values         =  9,/* [in]                              */    Fcgi_get_values_result  = ten/  * [out]                             */} fcgi_ Request_type;

The order in which messages are sent

is a simple message delivery process

The first to send is FCGI_BEGIN_REQUEST , and FCGI_PARAMS then FCGI_STDIN , because each message header (explained below) is capable of hosting a maximum length of 65535, so these two types of messages are not necessarily sent only once, and may be sent multiple times in succession.

After the FastCGI response body has been processed, it will be sent FCGI_STDOUT , FCGI_STDERR and in the same vein, multiple successive sends. Last to FCGI_END_REQUEST indicate the end of the request.

It is important to note that the FCGI_BEGIN_REQUEST FCGI_END_REQUEST start and end of the request are identified separately from the entire protocol, so the content of their message body is part of the agreement, so there will be corresponding structures corresponding to them (detailed later). environment variables, standard inputs, standard outputs, error outputs, which are business-related, are not protocol-independent, so the content of their message body does not correspond to the structure.

Since the entire message is binary continuous, it is necessary to define a unified structure of the message header so that the message body of each message is read to facilitate the cutting of the message. This is a very common means of communication in the network.

FastCGI message Header

As above, the FastCGI message is divided into 10 message types, some of which are output. All messages start with a message header. Its structure is defined as follows:

typedef struct _FCGI_HEADER {    unsigned char version;    unsigned char type;    unsigned char requestIdB1;    unsigned char requestIdB0;    unsigned char contentLengthB1;    unsigned char contentLengthB0;    unsigned char paddinglength;    unsigned char reserved;} Fcgi_header;

The fields are explained below:

• versionIdentifies the FASTCGI protocol version.

• typeIdentifies the fastcgi record type, which is the general function of recording execution.

• requestIdIdentifies the fastcgi request to which the record belongs.

• contentLengthThe number of bytes recorded for the Contentdata component.

About the above xxB1 and xxB0 the protocol description: When the two adjacent structural components are named in addition to the suffix "B1" and "B0", it indicates that both components can be considered as a single number valued as B1<<8 + B0. The name of the single number is the name of the component minus the suffix. This Convention sums up the processing of a number represented by more than two bytes.

For example, the requestId contentLength maximum value represented in the protocol header is65535

#include <stdio.h> #include <stdlib.h> #include <limits.h>int main () {   unsigned char requestIdB1 = Uchar_max;   unsigned char requestIdB0 = Uchar_max;   printf ("%d\n", (requestIdB1 << 8) + requestIdB0); 65535}

You might think that if a message body length exceeds 65535, then the split is sent to multiple messages of the same type.

Definition of Fcgi_begin_request

typedef struct _FCGI_BEGIN_REQUEST {    unsigned char roleB1;    unsigned char roleB0;    unsigned char flags;    unsigned char reserved[5];} Fcgi_begin_request;

Field interpretation

roleRepresents the role that the Web server expects the app to play. is divided into three roles (and what we're discussing here is typically the responder role)

typedef enum _FCGI_ROLE {    Fcgi_responder    = 1,    fcgi_authorizer    = 2,    fcgi_filter        = 3} fcgi_ Role

While FCGI_BEGIN_REQUEST the flags component in contains a bit that controls the line shutdown: flags & FCGI_KEEP_CONN if 0, the application closes the line after responding to this request. If not 0, the application does not close the line after responding to this request, and the Web server remains responsive to the line.

Definition of Fcgi_end_request

typedef struct _FCGI_END_REQUEST {    unsigned char appStatusB3;    unsigned char appStatusB2;    unsigned char appStatusB1;    unsigned char appStatusB0;    unsigned char protocolstatus;    unsigned char reserved[3];} Fcgi_end_request;

Field interpretation

appStatusA component is an application-level status code.
protocolStatusThe component is a status code at the protocol level, protocolStatus and the value may be:

Fcgi_request_complete: The normal end of the request.
Fcgi_cant_mpx_conn: Reject new request. This occurs when the Web server sends concurrent requests to the application through a line, which is designed to process one request at a time per line.
Fcgi_overloaded: Reject new request. This occurs when the app runs out of certain resources, such as a database connection.
Fcgi_unknown_role: Reject new request. This occurs when the Web server specifies a role that the application does not recognize.

protocolStatusThe definitions in PHP are as follows

typedef enum _FCGI_PROTOCOL_STATUS {    Fcgi_request_complete    = 0,    fcgi_cant_mpx_conn        = 1,    fcgi_ OVERLOADED            = 2,    fcgi_unknown_role        = 3} dcgi_protocol_status;

It is important to note that dcgi_protocol_status fcgi_role the values of each element are defined in the FastCGI protocol, not PHP custom.

Examples of message communication

For simplicity, the message header displays only the type of message and the ID of the message, and no other fields are displayed. The following example is from the official website

{fcgi_begin_request,   1, {fcgi_responder, 0}}{fcgi_params,          1, ' \013\002server_port80\013\016server_ ADDR199.170.183.42 ... "}{fcgi_stdin,           1," quantity=100&item=3047936 "}{fcgi_stdout,          1," content-type:text/html\r\n\r\n< Html>\n
With each of the above structures, you can think about the parsing and response process of the FastCGI responder:

First reads the message header, obtains its type FCGI_BEGIN_REQUEST , then parses its message body, learns that its need role is FCGI_RESPONDER , is flag 0, indicates closes the line after the request completes. Then parse the second message, know that the message type is FCGI_PARAMS , and then directly in the body of the message with a carriage return cut into the environment variable. Similarly, after processing is complete, the FCGI_STDOUT message body and the FCGI_END_REQUEST message body are returned for the WEB server to resolve.

The implementation of FastCGI in PHP

The following interpretation of the code notes is only a refinement of my personal knowledge, if there are errata, please point out. Not familiar with the code of the students may be a guide, preliminary understanding, if you feel very vague unclear, then still need to go through the line to read.

As an php-src/sapi/cgi/cgi_main.c example, assume that the development environment is a UNIX environment. The definition of some variables in the main function, as well as the initialization of SAPI, we will not discuss here, only about FastCGI related content.

1. Open a Socket monitoring service

FCGI_FD = Fcgi_listen (Bindpath, 128);
Start listening from here, and the fcgi_listen function inside is the first three steps to complete the socket service, socket bind listen .

2. Initializing the Request object

fcgi_requestallocates memory for the object, binding the listening socket socket.

Fcgi_init_request (&request, FCGI_FD);
The entire request is made from input to return, all around the fcgi_request structure object.

typedef struct _FCGI_REQUEST {    int            listen_socket;    int            FD;    int            ID;    int            keep;    int            closed;    int            In_len;    int            In_pad;    Fcgi_header   *out_hdr;    unsigned char *out_pos;    unsigned char  out_buf[1024*8];    unsigned char  reserved[sizeof (FCGI_END_REQUEST_REC)];    HashTable     *env;} fcgi_request;
3. Create multiple CGI parser sub-processes

The number of child processes here defaults to 0, reading the settings from the configuration file to the environment variable, then reading in the program, and then creating a specified number of child processes to wait for the request to process the WEB server.

if (getenv ("Php_fcgi_children")) {    char * children_str = getenv ("Php_fcgi_children");    Children = atoi (CHILDREN_STR);    ...} do {    pid = fork ();    Switch (PID) {case    0: Parent        = 0;//change the child process ID to 0 to prevent looping fork        /* t catch our signals *        /sigaction (SIG term, &old_term, 0);        Sigaction (sigquit, &old_quit, 0);        Sigaction (SIGINT,  &old_int,  0);        break;    Case-1:        perror ("PHP (pre-forking)");        Exit (1);        break;    Default:/        * Fine */        running++;        break;    }} while (parent && (Running < children));
4. Receiving requests in a child process

Everything here is a socket service routine. The request is accepted and then called fcgi_read_request .

Fcgi_accept_request (&request)
int fcgi_accept_request (fcgi_request *req) {    int listen_socket = req->listen_socket;    sa_t sa;    socklen_t len = sizeof (SA);    REQ->FD = Accept (Listen_socket, (struct sockaddr *) &sa, &len);    ...    if (req->fd >= 0) {        //use multiplexing mechanism        struct POLLFD FDS;        int ret;        FDS.FD = req->fd;        fds.events = Pollin;        fds.revents = 0;        do {            errno = 0;            RET = Poll (&fds, 1, n);        } while (Ret < 0 && errno = = eintr);        if (Ret > 0 && (fds.revents & Pollin)) {break            ;        }        Just close the socket connection, do not empty req->env        fcgi_close (req, 1, 0);    }    ...    if (Fcgi_read_request (req)) {        return req->fd;    }}
requestIt is important to put the global variable sapi_globals.server_context in place, which facilitates the invocation of the request elsewhere.

SG (Server_context) = (void *) &request;
5. Reading data

The following code removes the handling of some exception cases, showing only the order in which the execution is performed normally.

The fcgi_read_request message read in the message communication sample is done in, and many of the len = (hdr.contentLengthB1 << 8) | hdr.contentLengthB0; operations have been explained in the previous FASTCGI message header.

Here is the key to parsing the FastCGI protocol.

Static inline ssize_t safe_read (fcgi_request *req, const void *buf, size_t count) {    int    ret;    size_t n = 0;    do {        errno = 0;        ret = Read (REQ->FD, ((char*) buf) +n, count-n);        n + = ret;    } while (n! = count);    return n;}
static int fcgi_read_request (Fcgi_request *req) {... if (Safe_read (req, &AMP;HDR, sizeof (fcgi_header))! = sizeof (FCG I_header) | |    Hdr.version < fcgi_version_1) {return 0; Len = (hdr.contentlengthb1 << 8) |    hdr.contentlengthb0;    padding = hdr.paddinglength;    Req->id = (hdr.requestidb1 << 8) + hdr.requestidb0;        if (Hdr.type = = Fcgi_begin_request && len = = sizeof (fcgi_begin_request)) {char *val;        if (Safe_read (req, buf, len+padding)! = len+padding) {return 0;        } Req->keep = (((fcgi_begin_request*) buf)->flags & Fcgi_keep_conn); Switch ((((fcgi_begin_request*) buf)->roleb1 << 8) + ((fcgi_begin_request*) buf)->roleb0) {case fcgi                _responder:val = Estrdup ("RESPONDER");                Zend_hash_update (req->env, "Fcgi_role", sizeof ("Fcgi_role"), &val, sizeof (char*), NULL);            Break ... default:rEturn 0;            } if (Safe_read (req, &AMP;HDR, sizeof (fcgi_header))! = sizeof (Fcgi_header) | | hdr.version < fcgi_version_1) {        return 0; Len = (hdr.contentlengthb1 << 8) |        hdr.contentlengthb0;        padding = hdr.paddinglength; while (Hdr.type = = Fcgi_params && len > 0) {if (Safe_read (req, &AMP;HDR, sizeof (fcgi_header))! = Si Zeof (fcgi_header) | |                Hdr.version < fcgi_version_1) {req->keep = 0;            return 0; Len = (hdr.contentlengthb1 << 8) |            hdr.contentlengthb0;        padding = hdr.paddinglength; }        ...    }}
6. Execute the Script

Assuming this request is the PHP_MODE_STANDARD php_execute_script execution php file will be called. It's not going to unfold here.

7. Closing the request

Fcgi_finish_request (&request, 1);
int Fcgi_finish_request (fcgi_request *req, int force_close) {    int ret = 1;    if (req->fd >= 0) {        if (!req->closed) {            ret = Fcgi_flush (req, 1);            req->closed = 1;        }        Fcgi_close (req, force_close, 1);    }    return ret;}
fcgi_finish_requestcalled in fcgi_flush , fcgi_flush encapsulates a FCGI_END_REQUEST message body, and then safe_write writes the socket connection to the client descriptor.

8. Processing of standard input standard output

Standard input and standard output in the above not discussed together, actually in the cgi_sapi_module structure of the definition, but cgi_sapi_module this sapi_module_struct structure and other code coupling too much, I do not have in-depth understanding, here to do a simple comparison, I hope other netizens to give advice, add.

cgi_sapi_moduleA read that sapi_cgi_read_post handles post data is defined in.

while (Read_bytes < count_bytes) {    fcgi_request *request = (fcgi_request*) SG (server_context);    Tmp_read_bytes = fcgi_read (request, buffer + read_bytes, count_bytes-read_bytes);    Read_bytes + = Tmp_read_bytes;}
The fcgi_read FCGI_STDIN data is read in the medium.
At the same time, it is cgi_sapi_module defined sapi_cgibin_ub_write to take over the output processing, which is called sapi_cgibin_single_write and finally realizes the FCGI_STDOUT encapsulation of the FastCGI packet.

Fcgi_write (Request, Fcgi_stdout, str, str_length);
Written in the last

The process of understanding the knowledge of FastCGI made such a note, the content of their understanding (self-thinking) in a structured manner, can make others easier to see and understand is also a difficult thing. At the same time also let oneself to this knowledge point of understanding and deeply a layer. There is still a lot of confusion in the understanding of PHP code Learning and I need to slowly digest and understand myself later.