Detailed description of the use of the Reactor event manager in the twisted framework of Python

Pave
In a lot of practice, it seems that we always use asynchronous programming in a similar way:

Monitoring Events

Event occurrence execution corresponding callback function

Callback complete (may result in new events added to the listener queue)

Back to 1, listen to events

So we call this asynchronous Pattern the reactor pattern, such as the run loop concept in iOS development, which is actually very similar to the reactor loop, which is the main thread's Run Loop Listener screen UI event that executes the corresponding event-handling code as soon as the UI event occurs. You can also generate events to the main thread execution by GCD, and so on.

Is the boost to the reactor model, twisted design is based on such a reactor mode, twisted program is waiting for events, processing events in the process of continuous circulation.

From twisted.internet import Reactorreactor.run ()

Reactor is a singleton object in the twisted program.

Reactor
Reactor is an event manager that is used to register, unregister, and run an event loop that invokes a callback function when an event occurs. There are several conclusions about reactor:

The reactor of twisted can only be started by calling Reactor.run ().

Reactor loops are run in the process in which they begin, that is, in the main process.

Once started, it will continue to run. The reactor will be under the control of the program (or under the control of a thread that starts it).

The reactor loop does not consume any CPU resources.

You do not need to create reactor explicitly, just introduce it OK.

The last one needs to be explained clearly. In twisted, reactor is singleton (that is, singleton mode), that is, there can be only one reactor in a program, and as soon as you introduce it, create one accordingly. This is the way the twisted is used by default, of course, twisted there are other ways to introduce reactor. For example, you can use system calls in Twisted.internet.pollreactor to poll instead of the Select method.

If you use a different reactor, you need to install it before introducing Twisted.internet.reactor. Here's how to install Pollreactor:

From twisted.internet import Pollreactorpollreactor.install ()

If you have introduced twisted.internet.reactor without installing other special reactor, twisted will install the default reactor based on the operating system. Because of this, the customary practice is not to introduce reactor in the topmost module to avoid installing the default reactor, but to install it in the area where you want to use reactor.
Here are the programs that use the Pollreactor rewrite above:

From twited.internet import Pollreactorpollreactor.install () from twisted.internet import Reactorreactor.run ()

So how does reactor achieve a single case? Take a look at what the from Twisted.internet import reactor did and understand.

Here is a partial code for twisted/internet/reactor.py:

# twisted/internet/reactor.pyimport Sysdel sys.modules[' twisted.internet.reactor ']from twisted.internet Import Defaultdefault.install ()

Note: All modules loaded into memory in Python are placed in Sys.modules, which is a global dictionary. When you import a module, you first look in the list to see if the module has already been loaded, and if it is loaded, simply add the name of the module to the namespace of the module that is calling import. If it is not loaded, find the module file from the Sys.path directory according to the module name, locate and then load the module into memory, add it to sys.modules, and import the name into the current namespace.

If we are running from twisted.internet import reactor for the first time, Because there is no twisted.internet.reactor in Sys.modules, the code in reactory.py is run and the default reactor is installed. After that, if the module is already present in Sys.modules, the twisted.internet.reactor in Sys.modules is imported directly into the current namespace.

Install in default:

# Twisted/internet/default.pydef _getinstallfunction (Platform): "" "  Return a function to install the reactor Most suited for the given platform.   @param platform:the Platform for which to select a reactor.  @type Platform:l{twisted.python.runtime.platform}   @return: A zero-argument callable which would install the selected    reactor.  "" " Try:    if Platform.islinux ():      try: From        twisted.internet.epollreactor import install      except Importerror: From        twisted.internet.pollreactor import install    elif platform.gettype () = = ' posix ' and not PLATFORM.ISMACOSX (): From      twisted.internet.pollreactor Import install    else:      from Twisted.internet.selectreactor Import Install  except Importerror: from    twisted.internet.selectreactor Import Install  return install  install = _getinstallfunction (platform)

Obviously, the default will be based on the platform to obtain the appropriate install. Linux will use the Epollreactor first, if the kernel is not supported, you can only use Pollreactor. The Mac platform uses pollreactor,windows with Selectreactor. The implementation of each type of install is similar, here we take the selectreactor in the install to see.

# Twisted/internet/selectreactor.py:def Install (): "" "  Configure the Twisted mainloop to be run using the Select () reac Tor.  "" " # Single Example  reactor = Selectreactor () from  twisted.internet.main import installreactor  installreactor (reactor) # Twisted/internet/main.py:def Installreactor (Reactor): "" "  Install Reactor c{reactor}.   @param Reactor:an Object, provides one, or more ireactor* interfaces.  "" # This stuff should is common to all reactors.  Import twisted.internet  Import sys  if ' twisted.internet.reactor ' in Sys.modules:    raise error. Reactoralreadyinstallederror ("Reactor already Installed")  Twisted.internet.reactor = Reactor  sys.modules [' twisted.internet.reactor '] = reactor

In Installreactor, add the Twisted.internet.reactor key to Sys.modules, and the value is the singleton reactor created in the install. To use reactor in the future, you will import this singleton.

selectreactor# Twisted/internet/selectreactor.py@implementer (Ireactorfdset) class Selectreactor (Posixbase. Posixreactorbase, _extrabase)

Implementer represents selectreactor implementation of the Ireactorfdset interface method, where Zope.interface is used, it is the implementation of the interface in Python, interested students can go to see.

The Ireactorfdset interface mainly describes how to get, add, and delete descriptors. These methods can know the meaning of the name, so I did not add comments.

# Twisted/internet/interfaces.pyclass Ireactorfdset (Interface):   def addreader (reader):   def addwriter ( Writer):   def removereader (reader):   def removewriter (writer): Def   RemoveAll ():   def getreaders ():   def getwriters (): Reactor.listentcp ()

The REACTOR.LISTENTCP () in the example registers a listener event, which is a method in the parent class posixreactorbase.

# Twisted/internet/posixbase.py@implementer (ireactortcp, IREACTORUDP, Ireactormulticast) class PosixReactorBase (_ Signalreactormixin, _disconnectselectablemixin, reactorbase): Def listentcp (self, port, factory, Backlog=50, Interface= "): P = tcp. Port (port, factory, backlog, interface, self) p.startlistening () return P # twisted/internet/tcp.py@implementer (inte Rfaces. Ilisteningport) class Port (base. Baseport, _socketcloser): Def __init__ (self, port, factory, Backlog=50, interface= ", Reactor=none):" "Initialize wit    h a numeric port to listen on. "" "base. Baseport.__init__ (self, reactor=reactor) Self.port = Port Self.factory = Factory Self.backlog = Backlog if abs Tract.isipv6address (interface): self.addressfamily = Socket.af_inet6 Self._addresstype = address. IPv6Address Self.interface = interface ... def startlistening (self): "" "Create and bind my socket, and begin list     Ening on it.     Create and bind sockets to start listening. This was called on UnseriaLization, and must is called after creating a server to begin listening on the specified port. "" "If Self._preexistingsocket is None: # Create a new socket and make it listen try: # creates socket s KT = Self.createinternetsocket () if self.addressfamily = = Socket.AF_INET6:addr = _resolveipv6 (self.interf Ace, Self.port) else:addr = (Self.interface, self.port) # bound Skt.bind (addr) except Soc    Ket.error as Le:raise cannotlistenerror (Self.interface, Self.port, le) # monitoring Skt.listen (Self.backlog)       Else: # Re-use the externally specified socket Skt = self._preexistingsocket Self._preexistingsocket = None      # Avoid shutting it down at the end. Self._shouldshutdown = False # Make sure so if we listened on Port 0, we update this to # reflect what the OS act    Ually assigned us. Self._realportnumber = Skt.getsockname () [1] log.msg ("%s starting on%s"% (self._getlOgprefix (self.factory), Self._realportnumber) # The order of the next 5 lines is kind of bizarre.    If No one # can explain it, perhaps we should re-arrange them. Self.factory.doStart () self.connected = True Self.socket = Skt Self.fileno = Self.socket.fileno SELF.NUMBERACC epts = # startreading calls Reactor's Addreader method to add port to the Read Collection self.startreading ()

The whole logic is simple, like the normal server side, to create sockets, bindings, listening. The difference is that the socket descriptor is added to the Read collection of reactor. Then if the client is connected, reactor will monitor it and then trigger the event handler.

Reacotr.run () event main loop

# Twisted/internet/posixbase.py@implementer (ireactortcp, IREACTORUDP, Ireactormulticast) class PosixReactorBase (_ Signalreactormixin, _disconnectselectablemixin, reactorbase) # Twisted/internet/base.pyclass _SignalReactorMixi N (object): Def startrunning (self, Installsignalhandlers=true): "" "Posixreactorbase parent class _signalreactormixin and Reactorb    The ASE has this function, but _signalreactormixin before, installing the MRO order, will first call in _signalreactormixin. "" "Self._installsignalhandlers = Installsignalhandlers reactorbase.startrunning (self) def run (self, installsignal     Handlers=true): self.startrunning (installsignalhandlers=installsignalhandlers) self.mainloop () def mainLoop (self):          While Self._started:try:while self._started: # Advance simulation time in delayed event          # processors. Self.rununtilcurrent () t2 = self.timeout () t = self.running and T2 # doiteration is the key, Select,poll , Epool implementations have different self.doiteration (t) except:        Log.msg ("Unexpected error in main loop.") Log.err () else:log.msg (' Main loop terminated. ')

Mianloop is the final main loop, in the loop, call the Doiteration method to monitor the collection of read and write descriptors, once the descriptor is found ready to read and write, the corresponding event handlers are called.

# Twisted/internet/selectreactor.py@implementer (Ireactorfdset) class Selectreactor (Posixbase. Posixreactorbase, _extrabase): Def __init__ (self): "" "Initialize file descriptor tracking dictionaries and the BA    Se class. "" "Self._reads = set () Self._writes = set () posixbase.     Posixreactorbase.__init__ (self) def doselect (self, timeout): "" "Run one iteration of the I/O monitor loop.    This would run all selectables the had input or output readiness waiting for them.                  "" "Try: # Call the Select method to monitor the read-write collection, return the ready-to-Read descriptor R, W, ignored = _select (Self._reads, Self._writes,      [], timeout) except ValueError: # Possibly a file descriptor has gone negative? Self._preendescriptors () return except TypeError: # Something *totally* Invalid (object w/o Fileno, Non-integ RAL # Result) was passed Log.err () self._preendescriptors () return except (Select.error, Socket.err Or, IOError) as SE: # Select (2) encountered an error, perhaps while calling the Fileno () # Method of a socket.      (Python 2.6 socket.error is a IOError # subclass, but on Python 2.5 and earlier it's not.) If Se.args[0] in (0, 2): # Windows does this if it got a empty list if (not self._reads) and (not Self._wri TES): Return else:raise elif se.args[0] = = Eintr:return Elif Se.args[0] = = EBA Df:self._preendescriptors () return else: # OK, I really don ' t know what's going on.        Blow up. Raise _DRDW = Self._doreadorwrite _logrun = Log.callwithlogger for Selectables, method, Fdset in ((R, "Doread", Self._reads), (W, "Dowrite", Self._writes)): For selectable in Selectables: # If this is        Disconnected in another thread, kill it. # ^^ ^^---What's the!@#*? Serious! -exarkun if selectable not in Fdset:continue # this for pausing input when we ' re Not ready for more. # Call the _doreadorwrite method _logrun (selectable, _DRDW, selectable, method) Doiteration = Doselect def _doreadorwrite (s Elf, selectable, method): Try: # call Method,doread or Dowrite, # The selectable here is probably the TCP we're listening to. Port why = GetAttr (selectable, method) () except:why = Sys.exc_info () [1] log.err () if why:self. _disconnectselectable (selectable, why, method== "Doread")

Then, if the client has a connection request, it invokes TCP in the Read collection. Port's Doread method.

# twisted/internet/tcp.py @implementer (interfaces. Ilisteningport) class Port (base.    Baseport, _socketcloser): Def doread (self): "" "Called when my sockets are ready for reading.    Call this accepts a connection and calls Self.protocol () to handle the Wire-level protocol when the socket is ready to read.      "" "try:if platformtype = =" POSIX ": numaccepts = self.numberaccepts else:numaccepts = 1 For I in Range (numaccepts): If Self.disconnecting:return try: # Call Accept Skt, ad Dr = Self.socket.accept () except Socket.error as E:if E.args[0] in (Ewouldblock, eagain): Self . numberaccepts = i break elif e.args[0] = = Eperm:continue elif e.args[0] in (emfi LE, Enobufs, Enfile, Enomem, econnaborted): Log.msg ("Could not accept new connection (%s)"% (err   Orcode[e.args[0]],) break Raise Fdesc._setcloseonexec (Skt.fileno ())     protocol = Self.factory.buildProtocol (self._buildaddr (addr)) if Protocol is None:skt.close ()        Continue s = Self.sessionno Self.sessionno = s+1 # Transport The process of initialization, if it is in the read collection of reactor, then when it is ready        # When you're ready to read, you can call its Doread method to read the data sent by the client transport = Self.transport (Skt, protocol, addr, self, S, self.reactor) Protocol.makeconnection (transport) else:self.numberAccepts = self.numberaccepts+20 Except:log.defe RR ()

In the Doread method, the call accept generates a socket for receiving client data, binds the socket to transport, and then joins the transport to the read collection of reactor. When the client has data coming, it calls Transport's Doread method to read the data.

Connection is the parent class of the server (transport instance), which implements the Doread method.

# Twisted/internet/tcp.py@implementer (interfaces. Itcptransport, interfaces. Isystemhandle) class Connection (_tlsconnectionmixin, abstract. FileDescriptor, _socketcloser, _abortingmixin): Def doread (self): try: # receive data = Self.socket.re CV (self.buffersize) except Socket.error as se:if se.args[0] = = Ewouldblock:return Else:retur N Main. Connection_lost return self._datareceived (data) def _datareceived (self, data): If is not data:return main.       Connection_done # calls our custom protocol DataReceived method to process the data Rval = self.protocol.dataReceived (data) If Rval is not None:  offender = self.protocol.dataReceived Warningformat = (' returning a value other than None from% (FQPN) s  Is ' deprecated since% (version) s. ') warningstring = deprecate.getdeprecationwarningstring (offender, Versions.   Version (' Twisted ', one, 0, 0), Format=warningformat) deprecate.warnaboutfunction (offender, warningstring) Return Rval 

_datareceived called the example in our custom Echoprotocol datareceived method to process the data.

At this point, a simple process, from creating a listener event to receiving client data, is over.