"Technology sharing" Gateway Protocol learning: CGI, FastCGI, WSGI

Dg.

CGI, the Universal Gateway Interface (Common Gateway Interface), is the interface standard between external applications (CGI programs) and Web servers, and is a discipline for passing information between CGI programs and Web servers. The CGI specification allows the Web server to execute external programs and send their output to a Web browser, CGI turning a simple set of static hypermedia documents from the Web into a complete new interactive media. The popular CGI is like a bridge, which connects the Web page and the executing program in the Web server, passes the instruction that the HTML receives to the server's execution program, and returns the results of the server execution program to the HTML page. CGI has excellent cross-platform performance and can be implemented on almost any operating system.

The CGI method encounters a connection request (user request) to create a CGI subprocess, activate a CGI process, then process the request, and end the subprocess after processing. This is the Fork-and-execute mode. So how many CGI sub-processes are there for the CGI server, and the sub-process loading is the main reason for the poor CGI performance. When the user requests a very long time, will be heavily crowding out the system resources such as memory, CPU times, resulting in low performance.

CGI Scripting Workflow:

1. The browser requests a URL that points to a CGI application through an HTML form or hyperlink.
2. The server receives the request.
3. The server executes the specified CGI application.
4. CGI applications perform the required operations, usually based on what the browser has entered.
5. The CGI application formats the result as a Web server and a document that the browser can understand (usually an HTML page).
6. The Web server returns the results to the browser.

FastCGI

FastCGI is a scalable, high-speed interface for communicating between HTTP server and dynamic scripting languages. Most popular HTTP servers support FASTCGI, including Apache, Nginx, and lighttpd, while FASTCGI is supported by a number of scripting languages, including PHP.

FastCGI is an improvement from the development of CGI. The main disadvantage of the traditional CGI interface approach is poor performance because every time the HTTP server encounters a dynamic program, it needs to restart the script parser to perform the parsing, and the result is returned to the HTTP server. This is almost unusable when dealing with high concurrent accesses. FastCGI is like a resident (long-live) CGI, which can be executed all the time, so long as it is activated, it will not take a moment to fork once (this is the most notorious fork-and-execute mode of CGI). CGI is the so-called short-lived application, FastCGI is the so-called long-lived application. Because the FastCGI program does not need to constantly generate new processes, it can greatly reduce the pressure on the server and produce high application efficiency. It has a minimum speed of 5 times times higher than CGI technology. It also supports distributed operations where the FastCGI program can execute and accept requests from other Web servers on hosts other than the Web server.

FastCGI is a language-independent, extensible architecture for CGI open extensions whose main behavior is to keep the CGI interpreter process in memory and thus achieve high performance. As we all know, the repeated loading of CGI interpreter is the main reason of poor CGI performance, if the CGI interpreter remains in memory and accepts the FASTCGI process manager scheduling, it can provide good performance, scalability, fail-over characteristics and so on. FastCGI interface mode uses the C/s structure, can separate the HTTP server and the script parsing server, and start one or more script parsing daemon on the script parsing server. Each time the HTTP server encounters a dynamic program, it can be delivered directly to the fastcgi process to execute, and the resulting results are returned to the browser. This approach allows the HTTP server to handle static requests exclusively or return the results of the dynamic script server to the client, which greatly improves the performance of the entire application system.

FastCGI Workflow:

1. Load the FASTCGI process Manager (php-cgi or PHP-FPM or spawn-cgi) when Web server starts
2. The FASTCGI process Manager itself initializes, starts multiple CGI interpreter processes (visible multiple php-cgi) and waits for a connection from the Web server.
3. When a client request arrives at the Web server, the FASTCGI process manager selects and connects to a CGI interpreter. WEB server sends CGI environment variables and standard input to the FASTCGI child process php-cgi.
4. The FASTCGI child process returns standard output and error information from the same connection to the Web Server after processing is complete. When the fastcgi child process closes the connection, the request is processed to completion. The fastcgi child process then waits and processes the next connection from the FASTCGI process Manager (running in Web server). In CGI mode, php-cgi exits here.

Features of FastCGI

1. Break the traditional page processing technology. Traditional page processing technology, the program must be on the same server as the WEB server or the application server. This history has been broken by fastcgi technology, FASTCGI technology applications can be installed in the server farm of any one server, and through the TCP/IP protocol and the WEB server communication, this is suitable for the development of large distributed Web farms, but also suitable for efficient database control.
2. The explicit request pattern. CGI does not have a clear role, in the FastCGI program, the program is given a clear role (responder role, authenticator role, filter role).

Isapi

The ISAPI (Internet Server application program Interface) is a set of Web service-oriented API interfaces provided by Microsoft that implements all of the functionality provided by CGI, and expands on this basis, such as providing a filter application interface. Most ISAPI applications are used in the form of DLL dynamic libraries, can be executed after a user request, do not disappear immediately after processing a user request, but continue to reside in memory waiting to be processed by other user input. In addition, ISAPI DLL applications and Web servers are in the same process and are significantly more efficient than CGI. (due to the exclusivity of Microsoft, only running in the Windows environment)

The ISAPI server extension provides another option for Common Gateway Interface (CGI) applications that use Internet servers. Unlike CGI applications, ISA runs in the same address space as the HTTP server and can access all resources that can be used by the HTTP server. ISA has a lower system overhead than CGI applications because they do not require other processes to be created and do not perform traffic that needs to cross the process boundaries, which is time consuming. If memory is required by other processes, both the extension and the filter DLL may be unloaded. ISAPI allows multiple commands in a DLL, which are implemented as member functions of the CHttpServer object in a DLL. CGI requires that each task have a separate name and a URL mapping to a separate executable file. Each new CGI request initiates a new process, and each different request is contained in its own executable file, which is loaded and unloaded on a per-request basis, so the system overhead is higher than ISA.

php-cgi

PHP-CGI is the fastcgi manager that comes with PHP. PHP-CGI deficiency:

1. php-cgi Change php.ini configuration requires restart php-cgi for new Php-ini to take effect, no smooth restart
2. Directly kill the php-cgi process PHP will not be able to run. (PHP-FPM and spawn-fcgi do not have this problem, the daemon will smooth the new child process from the newly generated.) ）

spawn-fcgi

SPAWN-FCGI is a general-purpose fastcgi Management Server, it is a part of LIGHTTPD, many people use lighttpd spawn-fcgi for fastcgi mode management work, but there are many shortcomings. And the php-fpm of how much to alleviate some of the problems, but PHP-FPM has a drawback is to recompile, which for some already running environment may have a small risk, in PHP 5.3.3 can be directly used php-fpm. spawn-fcgi code is very small, all only 630 lines, in C language, the most recent submission was 5 years ago. Code Home: https://github.com/lighttpd/spawn-fcgi

SPAWN-FCGI code analysis is as follows:

1. spawn-fcgi first create Socket,bind,listen 3-Step server socket, (this socket is called FCGI_FD)
2. With Dup2, the FCGI_FD is exchanged to Fcgi_listensock_fileno (the Fcgi_listensock_fileno value equals 0, which is the socket ID specified for fastcgi in the Listen Protocol)
3. Execute EXECL, replaces the current process image with a new process image. Process image processes code snippet in run space

Obviously, spawn-fcgi is also a pre-fork model, just written in the ancient C language, full of n multi-Unix under the Dark programming skills.

The spawn-fcgi function is very single:

1. Just fork the process, the child process hangs, the main process log only once, do not fork at all. For some time in 2009, I used spawn-fcgi deployment php-cgi, when running for a period of time will be all hung up, only with crontab timed restart spawn-fcgi
2. Not responsible for the network IO in the sub-process, put the socket in the specified location is over, the next thing is handled by the Spawn program

SPAWN-FCGI is a very early procedure, and you can look at it. In addition: A section of the 1996 Code: HTTP://WWW.FASTCGI.COM/OM_ARCHIVE/KIT/CGI-FCGI/CGI-FCGI.C, and spawn-fcgi a style

php-fpm

PHP-FPM is a PHP fastcgi manager that is only for PHP and can be downloaded in http://php-fpm.org/download. PHP-FPM is actually a patch of PHP source code designed to integrate FASTCGI process management into a PHP package. It must be patch into your PHP source code and can be used after compiling and installing PHP. The FPM (FastCGI process Manager) is used to replace most of the additional features of php-cgi and is useful for high-load web sites. Its features include:

1. Advanced process management features that support smooth stop/start;
2. Can work in different uid/gid/chroot environments, and listen to different ports and use different php.ini configuration files (can replace Safe_mode settings);
3. StdOut and stderr log records;
4. Ability to restart and cache corrupted opcode in the event of an unexpected situation;
5. File upload optimization support;
6. Slow log – Record scripts (not only file names, but also PHP backtrace information that can be used by ptrace or similar tools to read and analyze the running data of remote processes), resulting in unusually slow operation;
7. Fastcgi_finish_request () – Special function: Used to continue to perform time-consuming work in the background (input video conversion, statistical processing, etc.) after request completion and refresh of data;
8. Dynamic/static sub-process generation;
9. Basic SAPI operation status information (Apache-like mod_status);
10. PHP.ini-based configuration file.

WSGI

The Web server gateway Interface (Python Interface, abbreviated as WSGI) is a simple and common interface between a Web server and a Web application or framework defined for the Python language. Similar interfaces have appeared in many other languages since Wsgi was developed. WSGI is a low-level interface between a Web server and a Web application or application framework to enhance the common denominator of portable web application development. WSGI is designed based on the existing CGI standards.

The Wsgi area is divided into two parts: one is "server" or "gateway" and the other is "application" or "Application Framework". When processing a WSGI request, the server provides the application with environmental information and a callback function (Callback functions). When the application finishes processing the request, the result is passed back to the server through the callback function described above. The so-called WSGI middleware implements both sides of the API, so it can mediate between the WSGI service and the WSGI application: From the perspective of the WSGI server, the middleware plays the application, and from the perspective of the application, the middleware acts as a server. The middleware component can perform the following functions:

1. After overriding the environment variable, the request message is routed to a different application object based on the destination URL.
2. Allows multiple applications or application frameworks to run concurrently in one process.
3. Load balancing and remoting, by forwarding request and response messages over the network.
4. Perform post-processing of content, such as applying an XSLT style sheet.

Previously, how to choose the right Web application framework became a problem for Python beginners because, in general, the choice of Web application framework would limit the choice of available Web servers and vice versa. At that time, Python applications were typically designed for one of the Cgi,fastcgi,mod_python, even for a custom API interface for a particular Web server. WSGI no official implementation, because WSGI is more like a protocol. The WSGI application (application) can run on any server (server), and vice versa, as long as these protocols are followed. WSGI is the CGI wrapper for Python, as opposed to fastcgi, which is a CGI wrapper for PHP.

WSGI divides Web Components into three categories: Web server, Web middleware, Web application, WSGI basic processing mode: WSGI Server--(WSGI middleware) *-WSGI application.

1, WSGI Server/gateway

Wsgi server can be understood as a WSGI compliant Web server that receives request requests, encapsulates a series of environment variables, invokes the registered Wsgi app according to the WSGI specification, and finally returns the response to the client. It's hard to explain exactly what WSGI server is, and what it does, the most intuitive way to look at the implementation code of WSGI server. As an example of Python's own wsgiref, Wsgiref is a simple WSGI server implemented in accordance with the WSGI specification. Its code is also not complex.

1. The server creates a socket, listens on the port, waits for the client to connect.
2. When a request comes in, the server parses the client information into the environment variable environ and invokes the bound handler to process the request.
3. Handler parses this HTTP request and places the request information such as Method,path into Environ.
4. Wsgi handler also put some server-side information into Environ, the last server information, client information, this request information is all saved to the environment variable environ.
5. WSGI handler calls the registered Wsgi app and passes Environ and callback functions to the Wsgi app
6. Wsgi app Reponse Header/status/body back to Wsgi handler
7. The final handler still plugs the response information back to the client via the socket.

2, WSGI Application

WSGI application is an ordinary callable object, and when a request arrives, Wsgi server calls the Wsgi app. This object receives two parameters, usually environ,start_response. Environ as described earlier, can be understood as an environment variable, all information related to a request is stored in the environment variable, including server information, client information, request information. Start_response is a callback function, WSGI application returns start_response response to Headers/status server by calling Wsgi. In addition, the Wsgi app will return a iterator object, the iterator is response body. It's so empty to say that it's a lot to see in this simple example.

3, WSGI Middleware

Some features may be between the server program and the application, for example, the server gets the URL of the client request, the different URLs need to be processed by different functions, this function is called URL Routing, this function can be placed in the middle of the implementation, this middle layer is middleware. Middleware is transparent to server programs and applications, meaning that the server program thinks it is the application and the application thinks it is the server. This tells us that middleware needs to disguise itself as a server, accept the application, invoke it, and middleware also need to disguise itself as an application to the server program.

In fact, whether it is a server program, middleware or application, all in the service side, to provide services to clients, the reason they are abstracted into different layers, is to control the complexity, so that each time is not too complex, their duties.

Resources:

• http://blog.csdn.net/on_1y/article/details/18803563
• http://blog.kenshinx.me/blog/wsgi-research/
• http://blog.ez2learn.com/2010/01/27/introduction-to-wsgi/

uwcgi

The UWSGI project is designed to develop a complete set of solutions for network applications that deploy distributed clusters. Uwsgi is mainly web-oriented and its standard service, which has been successfully applied in many different languages. Thanks to the extensible architecture of UWSGI, it can be extended to support more platforms and languages with unlimited extensions. Currently, you can use c,c++ and objective-c to write plugins. The "WSGI" in the project name is meant to thank the Python Web Standard with the same name because WSGI developed the first plug-in for the project. UWSGI is a Web server that implements protocols such as WSGI Protocol, UWSGI, HTTP, and so on. Uwsgi, without the WSGI protocol or FASTCGI protocol, but created a UWSGI protocol, UWSGI protocol is a UWSGI server-owned protocol, which defines the type of transmission information (type of information), each UWSGI Packet 4byte is a description of the transmission information type, which is two things compared to WSGI. The agreement is said to be about 10 times times as fast as the fcgi agreement.

1. The main features of Uwsgi are as follows:
2. Ultra-fast performance.
3. Low memory footprint (measured at about half of the Apache2 Mod_wsgi).
4. Multi-app management.
5. Detailed logging capabilities (can be used to analyze app performance and bottlenecks).
6. Highly customizable (memory size limit, service Restart after a certain number of times, etc.).

Other outreach knowledge: Java Servlet, Sinatra, Rack

Reprint Please specify: punctuation symbol» Gateway Protocol Learning: CGI, FastCGI, WSGI

Technology Sharing Gateway Protocol learning: CGI, FastCGI, WSGI