Python Web framework Tornado asynchronous Processing Code Demo sample

1. What is Tornado

Tornado is a lightweight but high-performance Python web framework, compared to Django, which also has a popular Python web framework. Tornado does not provide an ORM interface for manipulating databases and a rigorous MVC development model, but it can provide the primary Web server functionality. It is lightweight, and it is high-performance by using the non-blocking and Event-driven I/O model (Epoll or kqueue) to implement a set of asynchronous network libraries.

Tornado's Lightweight + high-performance features make it ideal for Web API applications where the non-clogging + async capability can be used to address c10k problems.

It is important to note that because the Python Gil causes multithreading to always be "characteristic" of a single-core operation, Tornado handles HTTP requests when the backend response of a request is blocked (such as reading data from a DB or disk causes very long processing time). Will cause other HTTP requests to be blocked, which can seriously slow down the performance of tornado in high concurrency scenarios.

Fortunately, it is. The tornado provides the ability to process requests asynchronously, in asynchronous mode, by passing in a callback function or by using the Tornado.gen.coroutine adorner provided by the tornado to make the IO inside the tornado Loop will still be able to accept other HTTP requests at the same time it waits for the results of the current request response, thus avoiding the processing power of a time-consuming operation affecting tornado.

2. How to write asynchronous processing code under the Tornado framework

Tornado official website document gives a few simple examples of asynchronous code demonstration, just to be honest, the code is too simple (all in a URI handler class get or post function to show the main asynchronous syntax), not much practical significance.

In the actual project. Complex processing logic is not likely to be stacked in a GET or post function, but instead encapsulated in other classes by a GET or post function call for the handler class.

Therefore, this paper gives a slightly more complex example to illustrate how to implement asynchronous processing logic in other class functions in order to achieve the purpose of asynchronous processing of HTTP requests.

If today's requirement is to implement a Web server with tornado, a URI method named Cityhotel is supported, and when the client visits the URI via an HTTP GET request, the Web server specifies the city according to the query number, There is also a back-end API to request that the hotel specific data be stored. After the business process, return a chain hotel to the client in all stores in the city.

If the URL format for the client GET request is: http://host/api/hotel/cityhotel?city=xxx
If the backend API to store the hotel specific data is:

City=xxx ">http://hotel_backend/getcityhotels?

City=xxx

According to the above scenario, since we implemented the Web server with tornado to the client request, there is also an API interface to request the underlying data, and the latter before returning, Tornado block, so, in such a scenario, Tornado is best to asynchronously request the API that provides the underlying data. Avoid an uncontrolled backend that drags Tornado response performance.

According to the business requirements described above. The following code demonstrates how to handle business processing asynchronously.

Module entry file (main.py):

#!/bin/env pythonImportTornado.ioloopImportTornado.webImportTornado.genImportHotelcore class cityhotelhandler(tornado.web.RequestHandler):    @tornado. Gen.coroutine     def get(self):        # # Parse query paramsparams = {} keys = [' City '] forKeyinchKeys:value = self.get_query_argument (key) Params[key] = value (status, RSP) =yieldHotelcore. Hotelapihandler.get_city_hotel (params[' City '])if  $= = Status:self.set_header (' Content-type ',' Application/json ') Self.finish (RSP)Else: Self.set_status (404) Self.finish () def main():App_inst = Tornado.web.Application ([(R '/api/hotel/cityhotel ', Cityhotelhandler),], Compress_response =True) App_inst.listen (8218) Tornado.ioloop.IOLoop.current (). Start ()if ' __main__ '= = __name__: Main ()

The module that handles the business logic is encapsulated in the hotelcore.py file, with code such as the following:

#!/bin/env python#-*-encoding:utf-8-*-ImportJson fromTornadoImportGen fromTornadoImportHttpClient class hotelapihandler(object):_cfg_dict = {' Api_host ':' api.hotelbackend.com ',    }@classmethod    @gen. Coroutine     def Get_city_hotel(CLS, city):RET =yieldCls._parallel_fetch_city_hotel (city)RaiseGen. Return (( $, ret))@classmethod    @gen. Coroutine     def _parallel_fetch_city_hotel(CLS, city):Base_url =' Http://%s/v1/getCityHotel '% (cls._cfg_dict[' Api_host '])# # Hote Type:1=normal, 2=deluxeHotel_type = {' normal ':1,' Deluxe ':2} urls = [] forVinchHotel_type.values (): Api_url ='%s?city=%s&level=%s '% (Base_url, city, v) urls.append (Api_url)# # Issue Async HTTP RequestHTTP_CLT = httpclient. Asynchttpclient () Rsps_dict =yieldDict (Normal_room = Http_clt.fetch (urls[0]), Deluxe_room = Http_clt.fetch (urls[1]) City_hotel_info = Cls._parse_city_hotel (rsps_dict, city) ret = {}ifLen (city_hotel_info): ret[' errno ']  =0ret[' ErrMsg '] =' SUCCESS 'ret[' Data '] = City_hotel_infoElse: ret[' errno ']  =1ret[' ErrMsg '] =' Service not Found @ 'ret[' Data ']   ="'        RaiseGen. Return (ret)@classmethod     def _parse_city_hotel(CLS, rsp_dict, City):City_hotel_info = {} forHotel_level, RSPinchRsp_dict.items (): Rsp_json = Json.loads (rsp.body) datas = rsp_json[' Data '] forCITY_ID, City_detailinchDatas.items (): name = city_detail[' name ']ifCityinchName:city_hotel_info[hotel_level] = City_detail Break        returnCity_hotel_info

A few additional notes on the above code:

• Writing tornado asynchronous processing code requires familiarity with Python's decorator syntax and Generator/yield syntax
• The adorner provided by Tornado @gen.coroutine indicates that the decorated function is an asynchronous handler, and that the function's call does not block tornado the main thread
• In the function that is decorated by @gen.coroutine, the time-consuming function that needs to run asynchronously is called by yield, and the yield itself returns a generator, combined with @gen.coroutine. It returns an object of the future type defined by the tornado
• The yield called function is in the process of running. Process Control is returned to the main thread, so even if the function requires a longer run time, the main thread of tornado can continue to process other requests
• In the syntax of the Python 2.x version number. The return value of the function is not agreed with return in generator. Must be provided by tornado raise Gen. Return (RET) achieves the purpose of returning. It's a tricky method.
• The future object returned by yield can get the return value of a function called through yield by calling the Body property
• Just to understand the syntactic meaning of @gen.coroutine and yield in tornado asynchronous programming, it is better to write complex asynchronous calling code than to write synchronous calling code that implements the same functionality but tornado overall performance is not guaranteed. The difficulty of implementation is almost nonexistent.

The above code is very many grammatical details are not unfolded, hope to realize the idea can help the people. ^_^

References

1. Tornado Doc:user ' s Guide
2. Book:introduction to Tornado Chapter 5. Asynchronous Web Services

Python Web framework Tornado asynchronous Processing Code Demo sample