"Go" analysis of the Django Framework architecture and Request/response processing process

This article is reproduced in crazy ants.

First, the core concept of the processing process

As shown in Django's overview map, take the following Django composition as a whole:

The core is that middleware Middleware,django all requests and returns are done by the middleware.

Middleware, is to deal with HTTP request and response, such as plug-ins, such as request middleware, view middleware, response middleware, exception middleware, middleware need in the "project/ settings.py "in the definition of middleware_classes. The approximate program flowchart is as follows:

First of all, middleware need to be in "project/settings.py" in the definition of middleware_classes, an HTTP request, will be specified by the middleware here to start from start to finish, for the time being called these need to process the middleware for processing chain, If a processor in the chain does not return response after processing, the request is passed to the next processor, and if a processor in the chain returns response, the direct out-of-process chain is processed by the response middleware and returned to the client, which can be called short-circuit processing. Second, the method in the middleware

The process of Django processing a request is done first through the middleware and then through the default URL. We can intercept all of the request in this place in middleware and return to response in our own way. Therefore, it is very necessary to understand the composition of middleware. Initializer: __init__ (self)

For performance reasons, each enabled middleware is initialized only once per server process. That is, __init__ () is called only when the service process is started, and is not executed against a single request processing.

For a middleware, the usual reason for defining the __init__ () method is the necessity of checking itself. If __init__ () throws an exception django.core.exceptions.MiddlewareNotUsed, Django will remove the middleware from the middleware stack.

When defining the __init__ () method in the middleware, you should not define any other parameters other than the standard self parameter. Request preprocessing function: Process_request (self, request)

This method is called immediately after the request is received by Django, but the URL is still not resolved to determine which view should run. Django passes the corresponding HttpRequest object to it so that it can be modified in the method.

Process_request () should return None or HttpResponse object.

If None is returned, Django will continue to process the request, execute the subsequent middleware, and then invoke the corresponding view.

If the HttpResponse object is returned, Django will no longer execute any other middleware (ignoring its kind) and the corresponding view. Django will return to the HttpResponse immediately. View preprocessing functions: Process_view (self, request, callback, Callback_args, Callback_kwargs)

This method is called when Django finishes executing the request preprocessing function and determines which view to execute, but before the view function actually executes.

Request

The HttpRequest object.

Callback

Django will invoke the Python function that handles the request. This is the actual function object itself, not the string representation of the function name.

Args

The position parameter list of the view is passed in, but not the request parameter (it is usually the first parameter passed into the view)
Kwargs

The keyword parameter dictionary that will pass in the view.

Like Process_request (), Process_view () should return None or HttpResponse objects.

If None is returned, Django will continue to process the request, execute the subsequent middleware, and then invoke the corresponding view

If the HttpResponse object is returned, Django will no longer execute any other middleware (regardless of the kind) and the corresponding view. Django will immediately return to the response post-processing function: Process_response (self, request, response)

This method is invoked after Django executes the view function and generates response.

The processor can modify the contents of the response; A common use is content compression, such as the HTML page requested by Gzip.

The parameters of this method are fairly straightforward: request is the request object, and response is the response object returned from view.

Process_response () must return the HttpResponse object. This response object can be the original object that passed in the function (which is usually modified), or it can be a completely new build. Exception post-processing function: Process_exception (self, request, exception)

This method is called only if there is a problem with the request process and the view function throws an uncaught exception. This hook can be used to send error notifications, output field-related information to a log file, or even attempt to recover automatically from an error.

The parameters of this function, in addition to the usual request object, include the actual exception object exception thrown by the view function.

Process_exception () should return None or HttpResponse object.

If None is returned, Django will continue to process the request using the framework's built-in exception handling mechanism.

If the HttpResponse object is returned, Django will use the response object, and the exception handling mechanism built into the short-circuit framework

The mechanism is described in detail in the following chapters. Third, the Django directory structure

Conf

There are two main functions: 1) Handle the global configuration, such as database, loaded application, middleware, etc. 2) processing URLs configuration, is the URL and view mapping relationship.

Contrib (contribution)

A functional module that is contributed by Django developers, but since it has been released with the release, it is official.

Core

Django's core processing library, including URL parsing, processing requests, caching, and so on, where processing requests are the core, such as handling fastcgi is handled by wsgi.py.

Db

As the name implies, dealing with a database is ORM.

Dispatch (Dispatch, Dispatch)

In fact, this is not Django original, is Pydispatch library, mainly dealing with consumer-worker mode.

Forms && Newforms && oldforms

A form that handles HTML, without introducing more than one.

Middleware

Middleware, is to handle HTTP request and response, like plug-ins. For example, a function of the default common middleware: When a page does not find the corresponding pattern, it will automatically add '/' re-processing. such as access to/blog, and the definition of the pattern is ' ^blog/$ ', so can not find the corresponding pattern, will automatically re-use/blog/find, of course, the premise is append_slash=true.

Template

Django templates

Templatetags

The wrapper to handle the application tag is to add all templatetags directories in the Installed_apps to the Django.templatetags directory, and when the tag is recorded using {{Load blog}} You can use the Import Django.templatetags.blog mode to load. However, there is a problem, if there are blog.py in other application directories, this will load the first tag that appears blog.py. In fact, there are many places in Django that need to deal with the same name, such as template, which requires extra care, and this follow-up is introduced.

Utils

Public libraries, many of the common classes are put here.

Views

The most basic view method. Iv. Django Terminology

In the application of Django, we often hear some terms:

Project

Refers to a complete Web service, typically composed of multiple modules.

Application

Can be understood as modules, such as user management, blog management, including the composition of data and data display, Applicaiton need to "project/settings.py" in the definition of Installed_apps.

Middleware

is the plug-in handling request and response, middleware need to middleware_classes in the "project/settings.py" definition.

Loader

The template loader, in fact, is in order to read the class of the template file, the default is through the file system loading and loading in the "application/templates" directory, loader need to "project/settings.py" in the template The definition of _loaders. V. Processing process

In fact, like other web frameworks, the process of HTTP processing is roughly the same,

The process of Django processing a request is done first through the middleware and then through the default URL. We can intercept all of the request in this place in middleware and return to response in our own way.

1. Load Configuration

The Django configuration is defined in "project/settings.py", either as a Django configuration or as a custom configuration, and is accessible through django.conf.settings.

2. Start

The core action is to start with the Command of django.core.management.commands.runfcgi, which runs the runfastcgi in django.core.servers.fastcgi, RUNFASTCGI uses the Flup wsgiserver to start the fastcgi. In Wsgiserver, an instance of the Django.core.handlers.wsgi Wsgihandler class is carried, which is handled by Wsgihandler to the request sent by the Web server (such as APACHE,LIGHTTPD, etc.), which is really going into Django's world.

3. Handling Request

When an HTTP request comes in, Wsgihandler begins to work, and it inherits from Basehandler. Wsgihandler creates an Wsgirequest instance for each request, and Wsgirequest is from HTTP. HttpRequest to inherit. The next step is to start creating Response.

4. Create response

The Get_response method of Basehandler is to create response according to request, and the action of generating response is to execute the corresponding view function in urls.py, which is also the one that Django can handle "friendly url". The key step, each of these functions, is to return a Response instance. It is common practice to load the template with loader and generate the page content, and it is important to extract the data from the database and render it into the template using ORM technology to generate a specific page.

5. Handling response

Django returns Response to Flup, Flup extracts the contents of the Response back to the WEB server, which is returned to the browser.

In summary, there are two main things that Django does in fastcgi: Handling Request and creating Response, and their corresponding core is "URLs analysis", "Template Technology" and "ORM Technology"

, an HTTP request is first converted to a HttpRequest object, which is then passed to the request middleware for processing, and if the middleware returns response, it is passed directly to the response middleware for finishing. Otherwise, the request middleware will access the URL configuration, determine which view to process, determine which view to execute, but when the view is not executed, the system will pass the request to the view middleware processor for processing, If the middleware returns response, then the response is passed directly to the response middleware for subsequent processing, otherwise the determined view function will be processed and returned to response, if an exception is thrown and thrown in the process, will be processed by the exception middleware processor. Vi. full flow of detail a Request arrived!

The first thing that happens is some other things that are related to Django (pre-preparation), namely:

1. If Apache/mod_python provides the service, request the Django.core.handlers.modpython.ModPythonHandler instance created by Mod_python to be passed to Django.

2. If it is a different server, you must be compatible with WSGI so that the server will create an Django.core.handlers.wsgi.WsgiHandler instance.

All two classes inherit from Django.core.handlers.base.BaseHandler, which contains the common code required for any type of request. There is a processor (Handler).

When one of the above processors is instantiated, a series of things happens immediately after:

1. This processor (handler) is imported into your Django configuration file.

2. This processor imports the Django custom exception class.

3. This processor calls its own Load_middleware method, loads all the middleware classes listed in the middleware_classes and introspect them.

The last one is a little complicated, let's take a closer look.

A middleware class can penetrate four stages of the processing process: Request,view,response and exception. To do this, you only need to define the specified, appropriate methods: Process_request,process_view, Process_response, and Process_exception. Middleware can define any one or all of these methods, depending on what functionality you want it to provide.

When the processor introspection middleware, it looks up the method of the above name and establishes four lists as the instance variables of the processor:

1. _request_middleware is a list of saved process_request methods (in each case, they are real methods that can be called directly), and these methods come from any middleware class that defines them.

1. _view_middleware is a list of saved Process_view methods that come from any of the middleware classes that define them.

2. _response_middleware is a list of saved process_response methods that come from any of the middleware classes that define them.

3. _exception_middleware is a list of saved process_exception methods that come from any of the middleware classes that define them. Green light: Start now

Now that the processor is ready to actually start processing, it sends a signal request_started to the scheduler (the Django internal Scheduler allows various components to declare what they are doing and can write code to listen for specific events.) There is no official documentation on this, but there are some comments on the wiki. ）。 Next it instantiates a django.http.HttpRequest subclass. Depending on the processor, it may be an instance of django.core.handlers.modpython.ModPythonRequest, or it may be django.core.handlers.wsgi.WSGIRequest An instance of the. Two different classes are required because Mod_python and WSGI APIs Pass in the request information in different formats, and this information needs to be parsed into a single standard format that Django can handle.

Once a httprequest or something like that exists, the processor calls its own Get_response method, passing in the HttpRequest as the only parameter. This is where almost all the real activity takes place. Middleware, first round.

The first thing Get_response do is traverse the processor's _request_middleware instance variable and invoke each of these methods, passing in the instance of the HttpRequest as a parameter.

For Middleware_method in Self._request_middleware:

Response = Middleware_method (Request)

If response:

Break

These methods can choose to short-circuit the remainder of the processing and immediately let Get_response return, by returning a value of its own (if they do, the return value must be an instance of Django.http.HttpResponse, which is discussed later). If one of them does this, we will immediately return to the main processor code, Get_response will not wait to see what other middleware classes want to do, it returns directly, and then the processor enters the response phase.

However, more generally, the middleware method applied here simply does some processing and decides whether to add, delete, or supplement the properties of the request. About parsing

Assuming there is no middleware directly returning response to the request, the processor next attempts to parse the requested URL. It looks for a configuration in the config file called root_urlconf, uses this configuration with the root URL/, as a parameter to create an instance of Django.core.urlresolvers.RegexURLResolver, and then calls its resolve method to resolve the requested URL path.

The URL resolver follows a fairly simple pattern. For each entry in the Urlpatterns list that is generated in the URL configuration file based on the configuration of root_urlconf, it checks whether the requested URL path matches the regular expression of the entry, and if so, there are two options:

1. If the entry has a callable include,resolver to intercept the matching URL, go to the URL configuration file that contains the specified and start traversing each entry in the Urlpatterns list. Depending on the depth and modularity of your URL, this may be repeated several times.

2. Otherwise, resolver returns three entries: The view function specified by the matching entry, an unnamed matching group from the URL (used as the location parameter for the view), and a keyword parameter dictionary, which consists of any named matching group from the URL and the URLConf The combination of any other keyword parameters obtained in the

Note that this process stops when it matches the first entry that specifies the view, so it is a good idea to have your URL configuration transition from a complex regular to a simple regular, which ensures that the resolver does not first match to the simple one and returns the wrong view function.

If no matching entries are found, resolver produces an django.core.urlresolvers.Resolver404 exception, which is a subclass of the django.http.Http404 exception. We'll see how it's handled later. Middleware, second round.

Once the desired view function and related parameters are known, the processor will look at its _view_middleware list and invoke the method, passing in the Httprequst,view function, for the view A list of positional parameters and a dictionary of keyword parameters.

# Apply View Middleware

For Middleware_method in Self._view_middleware:

Response = Middleware_method (Request, Callback, Callback_args, Callback_kwargs)

If response:

Break

Also, middleware is likely to intervene in this phase and force the processor to return immediately. Enter View

If the process continues at this point, the processor invokes the view function. The views in Django are not very strict because it only needs to meet a few conditions:

2 must be able to be called.

2 The instance of Django.http.HttpRequest must be accepted as the first-bit value parameter.

2 must be able to produce an exception or return an instance of Django.http.HttpResponse.

In addition to these, you can take the imagination. However, in general, views will use the Django database API to create, retrieve, update, and delete certain things from databases, and also load and render a template to render something to the end user. Template

The Django template system has two parts: part of the HTML used by designers to mix a small amount of other stuff, and the other part is to use pure Python for programmers.

From an HTML author's perspective, the Django template system is very simple and requires only three structures to know:

2 variable references. This is the case in the template: {{foo}}.

2 template filtering. In the example above, the filter bar is used as follows: {{Foo|bar}}. Typically this is used to format the output (e.g., run Textile, format date, and so on).

2 Template label. Is this: {% baz%}. This is where the "logic" of the template is implemented, you can {% if Foo%},{% for bar in foo%}, and so on, if and for are all template tags.

Variable references work in a very simple way. If you just want to print the variable, as long as {{Foo}}, the template system will output it. The only complication here is {{Foo.bar}}, when the template system tries several things sequentially:

1. First it tries a dictionary-style lookup to see if foo[' Bar ' exists. If it exists, its value is output, and the process ends with it.

2. If the dictionary lookup fails, the template system tries the property lookup to see if the foo.bar exists. It also checks to see if this property can be called, if it can be called.

3. If the property lookup fails, the template system attempts to find it as a list index.

If all of this fails, the template system output configures the value of Template_string_if_invalid, which by default is an empty string.

Template filtering is a simple Python functions that takes a value and a parameter and returns a new value. For example, date filtering uses a Python datetime object as its value, a standard strftime formatted string as its argument, and returns the result after the formatted string has been applied to the DateTime object.

Template tagging is a place where things are a little bit more complicated, and it's where you know how Django's templating system really works. The structure of the Django template

Internally, a Django template is represented as a "nodes" collection, which is inherited from the basic Django.template.Node class. Nodes can do all sorts of things, but there is one thing in common: each Node must have a method called Render, and it accepts the second argument (the first argument, apparently the Node instance) is an instance of Django.template.Context, which is a dictionary-like Object that contains the variables that all templates can get. Node's Render method must return a string, but if node's work is not output (for example, it is to modify the template context by adding, deleting, or modifying variables in the incoming context instance variable), you can return an empty string.

Django contains many sub-classes of Node to provide useful functionality. For example, each built-in template tag is processed by a Node subclass (for example, Ifnode implements the IF tag, Fornode implements the for tag, and so on). All built-in labels can be found in django.template.defaulttags.

In fact, all of the template structures described above are some form of Nodes, and plain text is not unusual. Variable lookup is handled by Variablenode, and for nature, filtering is also applied on Variablenode, where labels are Nodes of various types, and plain text is a textnode.

In general, a view renders a template with the following steps, which are:

1. Load the template you want to render. This is done by django.template.loader.get_template, which can use any of these methods to locate the desired template file. The Get_template function returns a Django.template.Template instance that contains the parsed template and the method used.

2. Instantiate a Context to render the template. If you are using the subclass Django.template.RequestContext of the context, the accompanying context handler automatically adds variables that are not defined in the view. The Context builder method uses a dictionary of key/value pairs (which becomes a name/value variable for the template) as its only argument, and RequestContext uses an instance of HttpRequest and a dictionary.

3. Call the Render method of the Template instance, the Context object as the first positional parameter.

The return value of the template's Render method is a string that is concatenated by the values returned by all the Nodes render methods in the template, in the order in which they appear in the template. About Response, a little

Once a template finishes rendering, or produces some other appropriate output, view is responsible for producing an Django.http.HttpResponse instance whose builder accepts two optional parameters:

1. A string that is the principal of the response (it should be the first positional parameter, or the content of the keyword parameter). Most of the time, this will be the output of rendering a template, but it is not necessary, where you can pass in any valid Python string.

2. The value of Content-type header as response (it should be the second position parameter or the keyword parameter mine_type). If you do not provide this parameter, Django will use the value of Default_mime_type in the configuration and the value of Default_charset, if you do not change them in the Django global configuration file, respectively, "text/html" and "Utf-8 ”。 Middleware, third round: exception

If the view function, or something in it, has an exception, then Get_response (I know we've spent some time digging into views and templates, but once the view returns or produces an exception, we'll still regain the get_response in the middle of the processor. method) will traverse its _exception_middleware instance variable and invoke each method there, passing in HttpResponse and this exception as a parameter. If it goes well, one of these methods instantiates a httpresponse and returns it. Still not responding?

At this time there may still not be a httpresponse, which may have several reasons:

1. View may not have a return value.

2. View may have an exception but no middleware can handle it.

3. A middleware method tries to handle an exception when it creates a new exception.

At this point, Get_response will return to its own exception handling mechanism, which has several levels:

1. If exception is Http404 and DEBUG is set to True,get_response will execute view django.views.debug.technical_404_response, incoming httpreques T and exception as parameters. This view shows the schema information that the URL resolver is trying to match.

2. If DEBUG is False and the exception is Http404,get_response calls the URL resolver resolve_404 method. This method looks at the URL configuration to determine which view is specified to handle the 404 error. The default is Django.views.defaults.page_not_found, but can be changed by assigning a value to the handler404 variable in the URL configuration.

3. For any other type of exception, if DEBUG is set to True,get_response, the view django.views.debug.technical_500_response will be executed, incoming HttpRequest and except Ion as a parameter. This view provides detailed information about the exception, including Traceback, local variables in each level stack, a detailed description of the HttpRequest object, and a list of all invalid configurations.

4. If DEBUG is False,get_response calls the URL Resolver's resolve_500 method, it is very similar to the resolve_404 method, when the default view is Django.views.default S.server_error, but can be changed by assigning a value to the handler500 variable in the URL configuration.

In addition, for any exception other than django.http.Http404 or Python built-in Systemexit, the processor sends a signal got_request_exception to the dispatcher, building a description of the exception before returning, Send it to everyone listed in the ADMINS configuration of the Django configuration file. Middleware, last round.

Now, regardless of the level at which the Get_response error occurs, it returns a HttpResponse instance, so we go back to the main part of the processor. Once it gets a httpresponse the first thing it does is traverse its _response_middleware instance variable and apply the method there, passing in HttpRequest and HttpResponse as arguments.

Finally

# Reset URLconf for this thread on the the-out

# Isolation of request.urlconf

Urlresolvers.set_urlconf (None)

Try

# Apply Response middleware, regardless of the response

For Middleware_method in Self._response_middleware:

Response = Middleware_method (Request, response)

Response = self.apply_response_fixes (Request, response)

Note that for any middleware who want to change something, this is their last chance. The check is in the mail

It's time to end it. Once the middleware completes the final link, the processor will send a signal to the dispatcher request_finished, which is definitely the last call to anything that wants to execute in the current request. The processor that listens for this signal empties and frees any resources that are in use. For example, a Django request_finished listener shuts down all database connections.

After this happens, the processor constructs an appropriate return value to send back to anything that instantiates it (now, is an appropriate mod_python response or a WSGI compatible response, depending on the processor) and returns.

It's over, from start to finish, that's how Django handles a request.

"Go" analysis of the Django Framework architecture and Request/response processing process