Python Web server Gateway Interface for building Web sites

This article is a Python build Web site series of the second article, followed by the above, mainly to tell you about the Web server Gateway Interface Wsgi related information, very detailed, the need for small partners can refer to the following

In the Web server and web framework where Python builds Web sites, we understand the concepts of Web servers, Web applications, and web frameworks. For Python, more and more web frameworks are available, giving us more choice and limiting our choice of Web Server. The same is true of Java, which has a lot of web frameworks, because there is a servlet API, and any Java Web framework-written application can run on any web Server.

The Python community certainly needs a set of APIs to fit the Web server and application, which is WSGI (Python Web server Gateway Interface), which is described in more detail in PEP 3333. To put it simply, Wsgi is a bridge between Web servers and Web applications, with the raw HTTP data taken from the Web server, processed into a unified format, handed to the Web application, and the business logic processed from the application/framework side, and the response content is delivered to the server.

The detailed process of coupling the Web server and the framework through WSGI is as follows:

WSGI Server Adaptation

The specific explanations are as follows:

The application (Network framework) provides a callable object named application (the WSGI protocol does not specify how to implement this object). Each time the server receives a request from an HTTP client, it invokes the callable object application, passing a dictionary called Environ as a parameter, and a callable object named Start_response. The framework/app generates HTTP status codes and HTTP response headers, and then passes them to Start_response, waiting for the server to be saved. In addition, the framework/app returns the body of the response. The server combines the status code, response header, and response body into an HTTP response and returns it to the client (this step is not part of the WSGI protocol).

Let's see how WSGI is adapted from the server side and the application side, respectively.

Server-side

We know that each HTTP request made by the client (usually the browser) consists of the request line, the message header, and the request body, which contains the details of the request. Like what:

Method: Indicates the methods performed on the resource identified by Request-uri, including Get,post, etc. user-agent: Allows the client to tell the server about its operating system, browser, and other properties;

After the server receives the HTTP request from the client, the WSGI interface must unify the request fields and pass them to the Application server interface (in fact, to the framework). What data the Web server specifically passes to the application, as it is described in CGI (Common Gateway Interface, Universal Gateway Interface), is called the CGI environment variable. WSGI inherits the content of the CGI environment variable, requiring the WEB server to create a dictionary to hold these environment variables (typically named Environ). In addition to the CGI-defined variables, environ must also save some WSGI-defined variables, and can also save some of the client system's environment variables, you can refer to Environ Variables to see what the specific variables.

Then the WSGI interface must give environ to the application to handle, here WSGI the application provides a callable object application, and then the server to call application, get the return value of the HTTP response body. When the server calls application, it needs to provide two variables, one is the variable dictionary environ mentioned earlier, and the other is the callable object Start_response, which produces the status code and the response header, so we get a full HTTP response. The WEB server returns the response to the client, and a complete HTTP request-response process is complete.

Wsgiref Analysis

Python has a built-in Web server that implements the Wsgi interface, and in module Wsgiref it is a reference implementation of a WSGI server written in pure Python, and we'll take a simple look at its implementation. First, let's say we start a WEB server with the following code:

# instantiate the server httpd = make_server (  ' localhost ', # The host name  8051,   # A port number where to wait  For the request  application  # The Application object name, in this case a function) # Wait for a single request, Serve it and quit Httpd.handle_request ()

We then take the Web server to receive a request, generate Environ, and then call application to process the request this main line to parse the source of the call process, simplified as shown:

WSGI Server Invocation Process

There are three main classes here, Wsgiserver,wsgirequesthandler,serverhandle. Wsgiserver is a Web server class that can provide Server_address (Ip:port) and Wsgirequesthandler classes to initialize to obtain a server object. The object listens on the port of the response, receives an HTTP request, creates an instance of the RequestHandler class through Finish_request, generates an instance of Handle class during the initialization of the instance, and then calls its run (application) function. The Application object provided by the application is then invoked inside the function to generate the response.

The inheritance relationships of these three classes are as follows:

WSGI class inheritance Diagram

Where TCPServer uses a socket to complete TCP communication, Httpserver is used to do HTTP-level processing. Similarly, Streamrequesthandler to handle stream Socket,basehttprequesthandler is used to deal with the HTTP level of content, which is not related to WSGI interface, more is the implementation of the WEB server, can be ignored.

Micro Server Instance

If the above wsgiref is too complex, the following one to implement a small Web server, so that we understand the Web server-side WSGI interface implementation. Code from self-developed Web server (ii), placed on the gist, the main structure is as follows:

Class Wsgiserver (object): # socket parameter address_family, Socket_type = socket.af_inet, socket. Sock_stream request_queue_size = 1 def __init__ (self, server_address): # TCP Server-side initialization: Creating sockets, binding addresses, listening ports # Get the servers address, Port de F Set_app (self, application): # Gets the application provided by the framework self.application = Application def serve_forever (self): # processing TC P connection: Get request content, call handler def handle_request (self): # Parse HTTP request, get environ, process request content, return HTTP response result env = Self.get_environ () res Ult = Self.application (env, Self.start_response) self.finish_response (Result) def parse_request (self, Text): # parse HTT   P Request Def Get_environ (self): # parse Environ parameter, here is just an example, there are a lot of parameters in fact. env[' wsgi.url_scheme ' = ' http ' ... env[' request_method '] = self.request_method # GET ... return env def start_re Sponse (self, status, Response_headers, Exc_info=none): # Add response header, status code self.headers_set = [Status, Response_headers + serv Er_headers] def finish_response (self, result): # returns HTTP response information server_address = (HOST, PORT) = ' ', 8888 # GenBuild a server Instance def make_server (server_address, application): Server = Wsgiserver (server_address) Server.set_app (application ) Return server

There are a lot of mature Web servers supporting WSGI, and Gunicorn is quite a good one. It originated from the unicorn of the Ruby community and was successfully ported to Python as a wsgi HTTP Server. Has the following advantages:

Easy configuration can automatically manage multiple worker processes select different background extension interfaces (sync, gevent, Tornado, etc.) application side (framework)

Compared to the server side, the application side (can also think of the framework) to do is much simpler, it only needs to provide a callable object (generally used to name it application), this object receives server-side pass two parameters environ and start_response. The callable object here can be not only a function, but also a class (the second example below) or an instance of the __call__ method, as long as the previous two parameters can be accepted, and the return value can be iterated by the server.

Application specifically to do is based on the information provided in the Environ on the HTTP request, a certain business processing, return an iterative object, the server side by iterating over this object, to obtain the body of the HTTP response. If there is no response body, then none can be returned.

At the same time, application also invokes the server-provided start_response, which generates the status code and Response header for the HTTP response, with the following prototype:

def start_response (self, status, Headers,exc_info=none):

Application needs to provide status: A string representing the HTTP response status string, and Response_headers: A list of tuples with the following form: (Header_name, Header_value), The headers used to represent the HTTP response. At the same time Exc_info is optional and is used for errors when the server needs to return information to the browser.

So far, we can implement a simple application, as follows:

def simple_app (environ, start_response): "" "  simplest possible application function" ""  Hello_world = "HELLO world!\n "  status = ' $ Ok '  response_headers = [(' Content-type ', ' Text/plain ')]  start_response (status, response_headers)  return [Hello_world]

Or use a class to implement the following.

Class Appclass: ""  produce the same output, but using a-Class "" "  def __init__ (self, Environ, start_response): 
  
   self.environ = environ   Self.start = start_response  def __iter__ (self):   ...   . Hello_world = "Hello world!\n"   yield Hello_world
  

Notice here that the Appclass class itself is application, called with Environ and start_response (instantiated) It returns an instance object, which itself is iterative and meets WSGI requirements for application.

If you want to use an object of the Appclass class as a application, you must add a __call__ method to the class, accept environ and start_response as arguments, and return the iterator object as follows:

Class Appclass: ""  produce the same output, but using a object "" "  def __call__ (self, Environ, start_response):

This section deals with some of the advanced features of Python, such as yield and magic method, which can be understood in my summary of Python language points.

The WSGI in Flask

Flask is a lightweight Python web framework that complies with WSGI specifications. Its original version is only more than 600 lines, relatively easy to understand. Let's look at the section on the WSGI interface in its original version.

def wsgi_app (self, Environ, start_response): "" "The  actual WSGI application.  This isn't implemented in ' __call__ ' So, middlewares can be applied:   App.wsgi_app = Mymiddleware (App.wsgi_app) 
   "" "With  Self.request_context (environ):   RV = self.preprocess_request ()   if RV is None:    RV = Self.dispatch_request ()   response = Self.make_response (RV)   response = self.process_response (response)   Return response (environ, start_response) def __call__ (self, Environ, start_response): ""  "Shortcut for:attr: ' Wsgi _app ' "" "  return Self.wsgi_app (environ, start_response)

The Wsgi_app here implements what we call the application function, RV is the encapsulation of the request, and response is the concrete function that the framework uses to handle the business logic. Here to flask source does not do too much explanation, interested can go to github download, and then check to the original version to see.

Middleware

In the comments in the previous Flask code Wsgi_app function, it is mentioned that the application part is not implemented directly in __call__, so that the middleware can be used. So why use middleware, what is middleware?

Recalling the previous application/server-side interface, for an HTTP request, the server side always calls a application for processing and returns the results after application processing. This is enough to deal with the general scenario, but not perfect, consider the following application scenarios:

For different requests (such as different URLs), the server needs to call different application, so how to choose which one to call; In order to do load balancing or remote processing, it is necessary to use application running on other hosts on the network for processing; The content returned by application needs to be processed before it can be used as an HTTP response;

One of the things that these scenarios have in common is that there are some required operations that are inappropriate either on the server side or on the application (frame) side. For the application side, these operations should be done by the server side, on the server side, these operations should be done by the application side. In order to deal with this situation, the middleware is introduced.

Middleware is like a bridge between the application and the service side, to communicate on both sides. On the server side, the middleware behaves like an application side, and it behaves like a server side to the application side. As shown in the following:

Middleware

Implementation of middleware

The Flask framework uses middleware in the initialization code of the Flask class:

Self.wsgi_app = Shareddatamiddleware (Self.wsgi_app, {self.static_path:target})

The role here, like the adorner in Python, is to perform some of the content in Shareddatamiddleware before and after executing Self.wsgi_app. Middleware does things that are similar to what the decorator does in Python. The Shareddatamiddleware middleware is provided by the Werkzeug Library to support site-managed static content. In addition, there are dispatchermiddleware middleware to support the invocation of different application depending on the request, which solves the problem in the previous scenario 1, 2.

Let's look at the implementation of Dispatchermiddleware:

Class Dispatchermiddleware (object): "" "allows one to  mount Middlewares or applications in a WSGI application.  This is useful if you want to combine multiple WSGI applications::   app = Dispatchermiddleware (app, {    '/app2 '):  APP2,    '/app3 ':  app3   })  "" "  def __init__ (self, app, mounts=none):   Self.app = App   Self.mounts = Mounts or {}  def __call__ (self, Environ, start_response):   script = Environ.get (' path_info ', ') C14/>path_info = "while   "/' in script:    if script in self.mounts:     app = Self.mounts[script]     break< C19/>script, Last_item = Script.rsplit ('/', 1)    path_info = '/%s%s '% (Last_item, path_info)   else:    app = SE Lf.mounts.get (script, self.app)   original_script_name = Environ.get (' script_name ', ')   environ[' Script_ NAME '] = original_script_name + script   environ[' path_info '] = path_info   return app (environ, start_response)

The middleware needs to be initialized with a mounts dictionary that specifies the mapping between different URL paths to application. So for a request, the middleware checks its path and then chooses the appropriate application for processing.

The basics of WSGI are basically over, and next I'll explain the understanding of the flask framework.