Python Learning---io async [custom Asynchronous IO]

Source: Internet
Author: User
Tags epoll httpcontext

Custom IO Async Basics:

--all requests are based on the socket implementation, and a request is a socket

Socket.setblocking (False) does not need to block, a request is over send another, will error, need to solve

--io multiplexing [is a synchronous request]

IO multiplexing has epoll, poll, select, knowing that epoll performance is better than others [Epoll find the house number first and look for someone].

Io multiplexing is essentially a way to perform multiple IO operations in the same thread or process by toggling the switch. Note that virtually every IO operation is performed independently. Just from the original one-to-one turn into many-to-many.

R,w,e = Select.select ([],[],[], Time-out) # The socket has a request/response to the previous r,w,e will receive the

io Async principle:

io Async implementation: essentially "non-blocking socket" + "IO multiplexing"

io Async implementation: essentially "non-blocking socket" + "IO multiplexing"

Import Selectimport socketimport timeclass asynctimeoutexception (timeouterror): "" "Request time-out exception class" "Def __init__ (s Elf, msg): Self.msg = Msg Super (asynctimeoutexception, self). __init__ (msg) # encapsulates the object class Httpcon for sockets and buffer        Text (object): "" "encapsulates the request and the corresponding basic data" "" Def __init__ (self, sock, host, port, method, URL, data, callback, timeout=5): "" "" Sock: Client socket object requested host: requested hostname port: Ports requested: Port method: Request Method URL        : Requested URL data: Callback in Request body: callback function after request completed timeout: timeout for request "" "Self.sock = Sock Self.callback = Callback Self.host = Host Self.port = Port Self.method = Method Self.url = URL Self.data = Data Self.timeout = Timeout Self.__start_time = time.time () Self.__buffer = [] def is_timeout (self): "" "whether the current request has timed out" "" Current_time = Time.time () if (Self.__start_time + self. Timeout) < Current_tiMe:return True def Fileno (self): "" "Request Sockect object's file descriptor for select Listener" "" Return Self.sock.fileno ()        Def write (self, data): "" In buffer Write response content "" "Self.__buffer.append (data) def finish (self, exc=none): "" "writes the response content in buffer complete, and executes the requested callback function" "If not Exc:response = B '. Join (Self.__buffer) self.cal Lback (self, Response, exc) else:self.callback (self, None, exc) def send_request_data (self): C        ontent = "" "%s%s Http/1.0\r\nhost:%s\r\n\r\n%s" ""% (Self.method.upper (), Self.url, Self.host, Self.data,)        Return Content.encode (encoding= ' UTF8 ') class Asyncrequest (object): Def __init__ (self): Self.fds = []        Self.connections = [] def add_request (self, host, port, method, URL, data, callback, timeout): "" Create a Request "" "  Client = Socket.socket () client.setblocking (False) # does not block the request, does not wait for the response Try:client.connect (host, Port)) except BlOckingioerror as E:pass # print (' request to send connection remotely ') req = HttpContext (client, host, port, method , URL, data, callback, timeout) self.connections.append (req) # Package HttpContext object Self.fds.append (req) def Check_conn_timeout (self): "" "checks for all requests, whether there is a connection timeout, if any, terminates" "" timeout_list = [] for the context in Self.connec             Tions:if context.is_timeout (): timeout_list.append (context) for the context in Timeout_list: Context.finish (asynctimeoutexception (' Request timed out ')) Self.fds.remove (context) self.connections.  Remove (context) def running (self): "" "Event loop to detect if the requested socket is ready to perform the relevant operation" "" while True:r, W, E            = Select.select (Self.fds, Self.connections, Self.fds, 0.05) if not Self.fds:return                        For-context in R:sock = Context.sock while true:try: data = SOCK.RECV (80) If not data:self.fds.remove (context) con                     Text.finish () break else:context.write (data)                         Except Blockingioerror as E:break except Timeouterror as E:                        Self.fds.remove (context) self.connections.remove (context)                 Context.finish (e) Break for context in W: # already connected to a successful remote server, start sending request data to remote If context in Self.fds:data = Context.send_request_data () context.s Ock.sendall (data) self.connections.remove (context) Self.check_conn_timeout () if __name__ = = '        __main__ ': Def callback_func (context, Response, ex): "" ":p Aram Context:httpcontext object, internally encapsulating request-related information :p Aram Response: Request Response Content        :p Aram ex: If an exception occurs (the value is the exception object if there is an exception; otherwise the value is None): return: "" Print (context, response, ex) obj = A Syncrequest () # based TCP constructs HTTP url_list = [{' Host ': ' www.google.com ', ' Port ': ', ' method ': ' GET ', ' url ': '/', ' Data ': ', ' timeout ': 5, ' callback ': Callback_func}, {' Host ': ' www.baidu.com ', ' Port ': ' method ': ' GET ' , ' url ': '/', ' data ': ', ' timeout ': 5, ' callback ': Callback_func}, {' Host ': ' www.bing.com ', ' Port ': ' Method ':        ' GET ', ' url ': '/', ' data ': ', ' timeout ': 5, ' callback ': Callback_func},] for item in Url_list:print (item) Obj.add_request (**item) obj.running ()

Python Learning---io async [custom Asynchronous IO]

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.