[Web server] (i) Tiny Web server Analytics

Written in front:

Plan to write a Web server, in the group's group Bo did not find the relevant articles, they intend to record this process from the beginning, one is to clean up my construction process, the second is also able to make the back of the students to do a reference.

Csapp on network Programming the chapter finally implements a small but full-featured Web server called Tiny. Because I just know some of the concepts of the HTTP protocol, it is not clear how the workflow and code organization of a Web server, and the book gives a complete implementation of the Tiny server, the code is very short, only hundreds of lines, so I imitated the beginning of the walk again, and try to analyze the code, run a bit, Give yourself an intuitive understanding. The source code is here, the annotated code is here. Next, analyze the tiny Web server.

The Ps:web Foundation does not write, oneself understands the basic concept, then looks the code to be enough.

Some of the common functions used in the Csapp above are placed in the Csapp.h header file and implemented in CSAPP.C. The function we see at the beginning of the capital letter is the addition of error handling to the original function function, such as

pid_t Fork (void) 
{  
     pid_t pid;

     if ((PID = fork ()) < 0)
         unix_error ("fork Error");
     return PID;
}  

(i) main function

Listen for connection requests on ports from the command line, open a listening socket through the OPEN_LISTENFD function, perform an infinite loop, accept connection requests continuously, perform HTTP transactions, and then turn off the connection after execution.

Tiny is single-threaded, and server cannot accept other customers when processing a client request, which is definitely not allowed in practical applications. Solution has

Multi-process: After accept fork, the parent process continues to accept, and the subprocess processes the CONNFD. So in high concurrency, there are several problems:
Question 1: Every time you come to a connection, fork overhead is too high. You can check what the system does when you call fork, and notice the copy of the parent Process page table.
Problem 2: After the concurrency, the process scheduler pressure is too high, process switching overhead is very large.
Problem 3: Under high load, consumes too much memory, in addition to high concurrency, the overhead of interprocess communication can not be ignored.

Multithreading: Accept after the thread to process the connection. This solves the problem of fork, but problems 2 and 3 are still unresolved. Thread pool: Number of threads fixed. Thread pool introduction and C++11 implementation. This can solve the above several problems.

int main (int argc, char **argv)
{
    int listenfd, CONNFD, Clientlen;

    struct sockaddr_in clientaddr;

    if (argc!= 2) {
        fprintf (stderr, "Usage:%s <port>\n", argv[0]);
        Exit (1);
    }

    Port = atoi (argv[1]);
    LISTENFD = OPEN_LISTENFD (argv[1]);

    while (1) {
        Clientlen = sizeof (CLIENTADDR);
        CONNFD = Accept (LISTENFD, (SA *) &clientaddr, &clientlen);
        Doit (CONNFD);
        Close (CONNFD);
    }

(b) doit function

The doit function handles an HTTP transaction. First read and parse the request line, use the Rio_readlineb function, refer to the strong read and write with the Rio package. Next, parse out method, URI, Version,tiny only support get methods, and if other methods, call the Clienterror function to return an error message.

Tiny does not use any of the information in the request header, and then reads and ignores the headers.

The URI is then parsed to parse the URI into a filename and a CGI parameter string. And get the request is static content or dynamic content.

If this file is not found, send an error message to the client and return.

If you are requesting static content, then first confirm the normal file and determine if there is read permission, and if all OK, then call the Serve_static function to provide static content. Similarly, invoking the Serve_dynamic function provides dynamic content.

/* $begin doit */void doit (int fd) {int is_static;
    struct stat sbuf;
    Char Buf[maxline], Method[maxline], Uri[maxline], version[maxline];
    Char Filename[maxline], cgiargs[maxline];

    rio_t Rio;
    /* Read Request line and headers */RIO_READINITB (&rio, FD);                   Rio_readlineb (&rio, buf, maxline);       Line:netp:doit:readrequest sscanf (buf, "%s%s", method, Uri, version);
       Line:netp:doit:parserequest if (strcasecmp (method, ' get ") {//line:netp:doit:beginrequesterr
        Clienterror (fd, method, "501", "Not Implemented", "Tiny does not implement");
    Return                              }//line:netp:doit:endrequesterr Read_requesthdrs (&rio); Line:netp:doit:readrequesthdrs/* Parse URI from GET request */is_static = Parse_uri (u       RI, filename, cgiargs); Line:netp:doit:staticcheck if (STAT (filename, &AMP;SBUF) < 0) {//line:netp:doit:beginnotfound clienterror (fd, filename, "404", "
    Not found "," Tiny couldn ' t find this file ");
    Return }//line:netp:doit:endnotfound if (is_static) {/* Serve static CO Ntent */if (!) ( S_isreg (sbuf.st_mode)) | | ! (S_irusr & Sbuf.st_mode)) {//line:netp:doit:readable clienterror (fd, filename, "403", "Forbidden", "Tiny couldn" t read the file
        ");
    Return        Serve_static (FD, filename, sbuf.st_size); Line:netp:doit:servestatic} else {/* Serve dynamic content */if (! S_isreg (sbuf.st_mode)) | | ! (S_ixusr & Sbuf.st_mode))  {//line:netp:doit:executable clienterror (fd, filename, "403", "Forbidden", "Tiny couldn" t run the CGI
        Program ");
    Return            Serve_dynamic (FD, filename, Cgiargs); Line:netp:doit:servedynamic}}/* $enddoit * 

(iii) Clienterror function

Tiny does not have complete error handling, but can check for some obvious errors and send it to the client. The corresponding status code and state information are included in the response line, and an HTML file is included in the response body to explain the error to the browser user.

/* $begin clienterror/void clienterror (int fd, char *cause, Char *errnum, Char *shor

    Tmsg, Char *longmsg) {char buf[maxline], body[maxbuf];
    * Build the HTTP response body * * sprintf (body, "
 
(d) Read_requesthdrs function

Tiny does not use any of the information in the request header, which reads and prints the contents of the request header on the server side. The empty line of the terminating request header is composed of a carriage return and a newline character pair.

/* $begin Read_requesthdrs *
/void Read_requesthdrs (rio_t *rp) 
{
    char buf[maxline];

    Rio_readlineb (RP, buf, maxline);
    while (strcmp (buf, "\ r \ n")) {          //line:netp:readhdrs:checkterm
    rio_readlineb (RP, buf, maxline);
    printf ("%s", buf);
    }
    return;
}
* * $end Read_requesthdrs * *
 
(v) Parse_uri function

Tiny assumes that the home directory of static content is the current directory, and that the home directory of the executable file is./cgi-bin/. Any URI that contains a string cgi-bin is considered a request for dynamic content. The default filename is./home.index.

In the case of static content, the CGI parameter string is known, the URI is converted to a relative pathname, and if the URI is at/end, the default filename is added.

If the content is dynamic, the CGI parameter is extracted and the remainder is converted to the relative filename.

/* $begin Parse_uri */int Parse_uri (char *uri, Char *filename, char *cgiargs) {char *ptr;                             if (!strstr (URI, "Cgi-bin")) {/* Static content///line:netp:parseuri:isstatic strcpy (Cgiargs, "");                           line:netp:parseuri:clearcgi strcpy (filename, ".");                           Line:netp:parseuri:beginconvert1 strcat (filename, URI);
        Line:netp:parseuri:endconvert1 if (Uri[strlen (URI)-1] = = '/')//line:netp:parseuri:slashcheck               strcat (filename, "home.html");
    Line:netp:parseuri:appenddefault return 1;                           else {/* Dynamic content///line:netp:parseuri:isdynamic ptr = index (URI, '? ');
        Line:netp:parseuri:beginextract if (PTR) {strcpy (Cgiargs, ptr+1);
    *ptr = ' n ';                         else strcpy (Cgiargs, ""); Line:netp:parseuri:endextract strcpy (filename,".");                           Line:netp:parseuri:beginconvert2 strcat (filename, URI);
    Line:netp:parseuri:endconvert2 return 0; }/* $end Parse_uri * *
 
(vi) serve_static function

Tiny provides four of different types of static content: HTML files, unformatted text files, and pictures encoded in GIF and JPEG format.

The Serve_static function sends an HTTP response that the subject is the content of the requested local file. The file type is first judged according to the filename suffix, and the response row and response header are sent, and the header is terminated with a blank line.

The next step is to send the response body. Here the Mmap function is used to map the requested file to a virtual storage space, after which the file can be manipulated through the pointer, and finally the mapped virtual memory area is released. About Mmap Please refer to: Careful analysis mmap: is what why how to use.

* * Serve_static-copy a file back to the client/* $begin serve_static/void serve_static (int fd, char *filenam
    e, int filesize) {int srcfd;

    Char *SRCP, Filetype[maxline], buf[maxbuf];       /* Send Response headers to client * * Get_filetype (filename, filetype);    Line:netp:servestatic:getfiletype sprintf (buf, "http/1.0 ok\r\n");
    Line:netp:servestatic:beginserve sprintf (buf, "%sserver:tiny Web server\r\n", buf);
    sprintf (buf, "%scontent-length:%d\r\n", buf, FileSize);
    sprintf (buf, "%scontent-type:%s\r\n\r\n", buf, filetype);       Rio_writen (FD, buf, strlen (BUF));    Line:netp:servestatic:endserve/* Send response body to client * * SRCFD = Open (filename, o_rdonly, 0);
    Line:netp:servestatic:open SRCP = Mmap (0, FileSize, Prot_read, Map_private, SRCFD, 0);//line:netp:servestatic:mmap                           Close (SRCFD);         Line:netp:servestatic:close Rio_writen (FD, SRCP, filesize); Line:netp:sErvestatic:write Munmap (SRCP, filesize); LINE:NETP:SERVESTATIC:MUNMAP} * * get_filetype-derive file type from file name/void Get_filetype (char *filenam
    E, char *filetype) {if (strstr (filename, ". html")) strcpy (filetype, "text/html");
    else if (strstr (filename, ". gif")) strcpy (filetype, "image/gif");
    else if (strstr (filename, ". jpg")) strcpy (filetype, "image/jpeg");
else strcpy (filetype, "text/plain"); }/* $end serve_static * *
 
(vii) serve_dynamic function

TINY derives a child process to run a CGI program to provide dynamic content.

This function starts by sending a response line that indicates success to the client.

The subprocess then initializes the QUERY_STRING environment variable with a CGI parameter from the request URI.

A CGI program sends its dynamic content to standard output and uses the DUP2 function to redirect standard output to the connected descriptor associated with the client before the child process loads and runs the CGI program. Anything written to the standard output by any CGI program will be sent directly to the client.

It then loads and runs the CGI program, in which the parent process blocks in the call to wait and waits for the resource that the operating system allocates to the child process when the child process terminates.

/* * Serve_dynamic-run a CGI program on behalf of the client * * $begin serve_dynamic/VO

    ID serve_dynamic (int fd, char *filename, char *cgiargs) {char buf[maxline], *emptylist[] = {NULL}; 
    /* Return to the HTTP response/* sprintf (buf, "http/1.0 ok\r\n");
    Rio_writen (FD, buf, strlen (BUF));
    sprintf (buf, "Server:tiny Web server\r\n");

    Rio_writen (FD, buf, strlen (BUF)); if (Fork () = = 0) {/* child *///line:netp:servedynamic:fork/* Real server would set all CGI VARs here/setenv ( "Query_string", Cgiargs, 1);         Line:netp:servedynamic:setenv Dup2 (FD, Stdout_fileno); /* Redirect stdout to client * *//line:netp:servedynamic:dup2 execve (filename, emptylist, environ); /* Run CGI Program *///line:netp:servedynamic:execve} wait (NULL); /* Parent waits for and reaps child *///line:netp:servedynamic:wait}/* $end serve_dynamic */
 
The above is an analysis of tiny code, with only hundreds of lines of C code to achieve a simple but effective Web server, which can provide both static content, but also to provide dynamic content. But building a full-featured and robust WEB server is not that simple, so there's a lot of detail to consider.

We've already said that using a thread pool solves several problems, but consider also that when dealing with a long connection, when a thread finishes processing a batch of data, it will read again, but there may not be data, because FD is blocked by default, so this thread will be blocked, when more threads are blocked, After more tasks came, it was still impossible to process. At this point, you can replace the blocking I/O to non-blocking I/O, when there is data readable to return the data, if there is no data readable on the return-1 and set the error to Eagain. How do you know when there is data to be read on FD? You can't always do polling in user mode ... Therefore, I am using the I/O multiplexing, an event-driven approach, so the recommended approach, non-blocking and IO multiplexing, is combined.

So the current plan implementation is this: IO multiplexing + non-blocking + threadpool design scheme.