Detailed explanation of the source code required for efficient tornado development, and detailed explanation of tornado source code

Detailed explanation of the source code required for efficient tornado development, and detailed explanation of tornado source code

This blog post focuses on tornado source code analysis. I believe that after reading this article, you can gain a deeper understanding of tornado's operating mechanism and use the tornado framework more efficiently.

First knowledge of tornado

First, let's start with the classic hello world case:

import tornado.ioloopimport tornado.web class MainHandler(tornado.web.RequestHandler):    def get(self):        self.write("Hello, world") application = tornado.web.Application([    (r"/index", MainHandler),]) if __name__ == "__main__":    application.listen(8888)    tornado.ioloop.IOLoop.instance().start()

Run the script and execute it in sequence:

• Create an Application object and pass a regular expression '/' and the class name MainHandler to the constructor tornado. web. Application (...)
• Execute the listen (...) method of the Application object, that is, application. listen (8888)
• Execute the start () method of the IOLoop class, that is, tornado. ioloop. IOLoop. instance (). start ()

1. Analyze the following code:

application = tornado.web.Application([    (r"/index", MainHandler),])

Source code:

From the source code of the Application class, we can see that the first parameter to be passed in is handlers, that is, the above Code can be expressed as follows:

application = tornado.web.Application(handlers=[    (r"/index", MainHandler),])

Parameters hereHandlersIt is very important and deserves further research. It should be a list of tuples. The first element of each tuple is a regular expression for matching, and the second element isRequestHanlderClass. InHello. pyOnly one regular expression-RequestHanlderYes, but you can specify any number as needed.

2. In-depth Application class:
Source code:

This is the description of the static file path in the source code, which is sequential from top to bottom:

• If the user configures a file path named "static_path" in settings (this requires that if we need to configure a static file path, the key value must be "static_path ")
• From the source code, we can see that this is based on the dictionary method, so we can conclude that settings is a dictionary format.
• This is to configure the static file path prefix, so that tronado can find the static file in the static file path at the front end, that is, tell tronado that I am a static file and query it by the static file path. Here, static file prefixes can also be understood as static file identifiers.

   

3. application. listen (8888 ):

 application.listen(8888)

That is, the listen method of the Application class:

Source code:

From the source code above, we can see that the listen method receives the port, which is the IP address. The default value is null. Then instantiate the HTTPServer object and execute the listen method:

HTTPServer listen method source code:

The listen method is bound to an ip address and port, and socket is enabled.

Bind method source code:

 

From the source code above, we can see that the bind method internally creates a socket object, calls the socket object to bind the ip address and port, and listens.

4. tornado. ioloop. IOLoop. instance (). start ():

tornado.ioloop.IOLoop.instance().start()

This is the ioloop class in the IOLoop. py file of tronado:

Source code of the instance method:

The class method is used here, that is, you can directly access it by class name. We need to pay attention to the final code:

if not hasattr(cls, "_instance"):            cls._instance = cls()        return cls._instance

Note: The Singleton mode is used here, because we do not need to create an object for each connection IO. In other words, each connection IO only needs to be the same object.

Start method:

Class IOLoop (object): def add_handler (self, fd, handler, events): # This method is called in the Start method of HttpServer. _ handlers [fd] = stack_context.wrap (handler) self. _ impl. register (fd, events | self. ERROR) def start (self): while True: poll_timeout = 0.2 try: # polling event_pairs = self in epoll. _ impl. poll (poll_timeout) except t Exception, e: # omit others # If read information is available, add the socket object handle and Event Code sequence to self. self. _ events. update (event_p Airs) # traverse self. _ events: process each request while self. _ events: fd, events = self. _ events. popitem () try: # Use socket as the handle key to retrieve self. in _ handlers, stack_context.wrap (handler) is executed, and # stack_context.wrap (handler) is a function that encapsulates the _ handle_events function of the HTTPServer class # It is added to self when the add_handler method is executed in the previous step. _ handlers. Self. _ handlers [fd] (fd, events) handle T: # omit other

From the above source code, while Ture: we can see that when the start method is executed, the program will enter an endless loop and constantly check whether a user sends a request. If a request arrives, execute the stack_context.wrap (handler) method that encapsulates the _ handle_events method of the HttpServer class and the relevant context (in fact, it is to execute the _ handle_events method of the HttpServer class). For details, refer to the following blog post:

class HTTPServer(object):    def _handle_events(self, fd, events):        while True:            try:                connection, address = self._socket.accept()            except socket.error, e:                if e.args[0] in (errno.EWOULDBLOCK, errno.EAGAIN):                    return                raise            if self.ssl_options is not None:                assert ssl, "Python 2.6+ and OpenSSL required for SSL"                try:                    connection = ssl.wrap_socket(connection,                                                 server_side=True,                                                 do_handshake_on_connect=False,                                                 **self.ssl_options)                except ssl.SSLError, err:                    if err.args[0] == ssl.SSL_ERROR_EOF:                        return connection.close()                    else:                        raise                except socket.error, err:                    if err.args[0] == errno.ECONNABORTED:                        return connection.close()                    else:                        raise            try:                if self.ssl_options is not None:                    stream = iostream.SSLIOStream(connection, io_loop=self.io_loop)                else:                    stream = iostream.IOStream(connection, io_loop=self.io_loop)                HTTPConnection(stream, address, self.request_callback,                               self.no_keep_alive, self.xheaders)            except:                logging.error("Error in connection callback", exc_info=True)

5. tornado. web. RequestHandler

This is the parent class that all business processing handler needs to inherit. Next, we will introduce some common methods in the RequestHandler class:

 

• Initialize:

From the source code, we can see that the initialize function will be executed during the RequestHandler class initialization, but the initialize function in the source code does not do anything. This is actually the place where tornado reserved for us to modify the source code, this allows the program to execute the methods defined in initialize before executing all handler.

• Write

Source code:

The write method is the method that the backend directly displays to the front-end page. From the source code above, we can see that the write method receives dictionary and string type parameters. If the data transmitted by the user is of the dictionary type, the source code will automatically serialize the dictionary using json, it is serialized into a string.

Self. _ write_buffer is a place defined in the source code to temporarily store the string to be output, in the List format.

• After the content to be written is added to self. _ write_buffer, the system will execute the flush method:

Flush method source code:

 

From the source code above, we can see that the flush method will be self. _ write_buffer all elements in the list are concatenated into strings, assigned to chunk, and then cleared. _ write_buffer list, set the request header, and finally call request. the write method is displayed on the front-end page.

• Render method:

Source code:

From the source code above, we can see that the render method is based on the parameter rendering template. Next we will introduce how to render the specific source code:

Source code of js and css:

From the source code above, we can see that the rendering process of static files (taking JavaScript as an example, css is similar) is:

First, use module. embedded_javascript () to obtain the JavaScript string to be inserted and add it to the js_embed list;

Then, use module. javascript_files () to obtain the List format JavaScript files, and add it to js_files.

Next, we will further introduce js_embed and js_files:

Js_embed source code:

The source code generates the script tag, which is some of our own JavaScript code. It is finally inserted into the entire html by concatenating strings.

Js_files source code:

 

 

The source code generates the script tag, which is the JavaScript code block to be introduced. It is inserted into the entire html By String concatenation. Note that the static path is implemented by calling self. static_url (path.

Static_url method source code:

The code above shows that the source code first determines whether the user has set the prefix of the static path, then Concatenates the static path and the relative path into an absolute path, and then opens the file according to the absolute path, and the file content (f. read () performs md5 encryption, and finally Concatenates the root directory + static path prefix + relative path in front-end html.

 

Render_string method:

This is the most important sub-method in the render method. It processes the Html template and returns the final result:

Detailed process:

 

 

Source code comment:

Class RequestHandler (object): def render_string (self, template_name, ** kwargs): # obtain the template folder path specified in the configuration file, that is, template_path = 'view' template_path = self. get_template_path () # if the path of the template file is not configured, find if not template_path: frame = sys in the directory where the program is started by default. _ getframe (0) web_file = frame. f_code.co_filename while frame. f_code.co_filename = web_file: frame = frame. f_back template_path = OS. path. dirname (frame. f_code.co_filename) If not getattr (RequestHandler, "_ templates", None): RequestHandler. _ templates ={}# create a Loader object. After the first creation, the value will be saved in the static field _ template_loaders of RequestHandler if template_path not in RequestHandler. _ templates: loader = self. application. settings. get ("template_loader") or \ template. loader (template_path) RequestHandler. _ templates [template_path] = loader # executes the load method of the Loader object. This method internally executes the _ create_template method of Loader _ The create_template method uses the open method internally to open the html file and read the html content. Then, it is used as a parameter to create a Template object # When the Template constructor is executed, parse the content of the html file internally, split the content according to the internal {}}{%} tag, generate a function represented by the string class, and save it in self. t = RequestHandler in the code field. _ templates [template_path]. load (template_name) # obtain all the values to be embedded in html and the default value args = dict (handler = self, request = self. request, current_user = self. current_user, locale = self. locale, _ = self. locale. translate, static_url = self. stati C_url, xsrf_form_html = self. xsrf_form_html, reverse_url = self. application. reverse_url) args. update (self. ui) args. update (kwargs) # execute the generate method of Template, compile the function represented by a string, set all keys and values in the namespace to global variables, and then execute this function. In this way, values are nested in html and returned. Return t. generate (** args)

Class Loader (object): "A template loader that loads from a single root directory. you must use a template loader to use template constructs like {% extends %} and {% include % }. loader caches all templates after they are loaded the first time. "def _ init _ (self, root_directory): self. root = OS. path. abspath (root_directory) self. templates = {} Loader. _ init __Loader. _ init _ class Loader (object): def load (self, name, parent_path = None): name = self. resolve_path (name, parent_path = parent_path) if name not in self. templates: path = OS. path. join (self. root, name) f = open (path, "r") # Read the content of the html file # create a Template object # name is the file name self. templates [name] = Template (f. read (), name = name, loader = self) f. close () return self. templates [name] Loader. loadLoader. loadclass Template (object): def _ init _ (self, template_string, name = "<string>", loader = None, compress_whitespace = None ): # template_string is the content of the Html file self. name = name if compress_whitespace is None: compress_whitespace = name. endswith (". html ") or name. endswith (". js ") # encapsulate the content in the _ TemplateReader object, which is used to split the html file reader = _ TemplateReader (name, template_string) according to the tag of the template language) # split html files into objects one by one # execute the _ parse method to split html files into _ ChunkList object self. file = _ File (_ parse (reader) # format the html content into a string representation of the function self. code = self. _ generate_python (loader, compress_whitespace) try: # compile the Function Represented by the string into the function self. compiled = compile (self. code, self. name, "exec") handle T: formatted_code = _ format_code (self. code ). rstrip () logging. error ("% s code: \ n % s", self. name, formatted_code) raiseTemplate. _ init __Template. _ init _ class Template (object): def generate (self, ** kwargs): "" Generate this template with the given arguments. "namespace = {" escape ": escape.xhtml _ escape," xhtml_escape ": escape.xhtml _ escape," url_escape ": escape. url_escape, "json_encode": escape. json_encode, "squeeze": escape. squeeze, "linkify": escape. linkify, "datetime": datetime,} # create a variable environment and execute the function. For details about the Demo, see namespace in the previous blog. update (kwargs) exec self. compiled in namespace execute = namespace ["_ execute"] try: # execute the compiled string format function to obtain the embedded html file return execute () failed t: formatted_code = _ format_code (self. code ). rstrip () logging. error ("% s code: \ n % s", self. name, formatted_code) raiseTemplate. generateTemplate. generate

Example html:

Source code template language processing:

Conclusion

This blog post analyzes the source code of tornado url regular matching, routing and ing, underlying socket implementation, port listening, multi-request concurrency, and Handler methods, it allows tronado web developers to have a deeper understanding of tronado's operating mechanism, so as to be more efficient in tronado web development.

If you think this article is of reference value to you, you are welcome to help the bloggers click the recommendation below the article. Thank you!