An Introduction to asynchronous programming and twisted (3)

Part 11: Your poetry is serveda twisted poetry Server

Now that we're 've learned so much about writing clients with twisted, Let's turn around and re-implement our poetry server with twisted too. and thanks to the generality of twisted's exactly actions, it turns out we 've already learned almost everything we need to know.

class PoetryProtocol(Protocol):     def connectionMade(self):        self.transport.write(self.factory.poem)        self.transport.loseConnection()class PoetryFactory(ServerFactory):        protocol = PoetryProtocol    def __init__(self, poem):        self.poem = poemdef main():    options, poetry_file = parse_args()    poem = open(poetry_file).read()    factory = PoetryFactory(poem)     from twisted.internet import reactor    port = reactor.listenTCP(options.port or 0, factory,                             interface=options.iface)      reactor.run()

It can be seen that the server and client are basically the same in principle. The reactor loop listens for the event and uses protocol for processing when the event arrives. The factory is used to manage protocol and inherits from serverfactory.

 

Part 12: A poetry transformation Server

In this section, a more complex server is implemented. Different requests are sent from the client, and poem is converted and sent back to the client. This requires a protocol for the client to communicate with the server normally.

Twisted includes support for several protocols we cocould use to solve this problem, includingXML-RPC,Perspective Broker, AndAmp.

But in order that our example is simple enough to make it easy to understand, we use our own simple protocol,

<Transform-Name>. <text of the poem>

After the server receives such a request from the client, it uses the transform-name to transform the text of the poem and sends it back to the client.

class TransformProtocol(NetstringReceiver):     def stringReceived(self, request):        if '.' not in request: # bad request            self.transport.loseConnection()            return         xform_name, poem = request.split('.', 1)         self.xformRequestReceived(xform_name, poem)     def xformRequestReceived(self, xform_name, poem):        new_poem = self.factory.transform(xform_name, poem)         if new_poem is not None:            self.sendString(new_poem)         self.transport.loseConnection()        class TransformFactory(ServerFactory):     protocol = TransformProtocol     def __init__(self, service):        self.service = service     def transform(self, xform_name, poem):        thunk = getattr(self, 'xform_%s' % (xform_name,), None)         if thunk is None: # no such transform            return None         try:            return thunk(poem)        except:            return None # transform failed     def xform_cummingsify(self, poem):        return self.service.cummingsify(poem)    class TransformService(object):     def cummingsify(self, poem):        return poem.lower()def main():    service = TransformService()    factory = PoetryFactory(service)     from twisted.internet import reactor    port = reactor.listenTCP(options.port or 0, factory,                             interface=options.iface)    reactor.run()

Let's take a look at this code,

First, transformprotocol inherits from netstringreceiver instead of protocol. netstringreceiver is a protocol dedicated to processing strings. Here you can use and inherit various protocols provided by the twisted development framework to simplify code, instead of starting from scratch, This is the benefit of using the framework.

In transformprotocol, for the specific transform logic of poem, call self. factory. transform, throwing the variable to the factory, while keeping the Protocol highly abstract, changing the transform logic, and adding or subtracting all keep the Protocol unchanged.

Second, in transformfactory, use Python's powerful getattr to avoid using a large number of if... Else.

However, only the cummingsify service is provided here. If you want to add or delete a service, transformfactory and transformservice will inevitably need to be modified...

This code has been well written... but it lacks some twisted feeling... if we add the deferred callback mechanism, we should be able to write more highlevel code.

 

Part 13: Deferred all the way downintroduction

Recall poetry client 5.1 from Part 10.the client used a deferred to manage a callback chain that was ded a call to a poetry transformation engine. In client 5.1, the engine was implemented asSynchronousFunction call implemented in the client itself.

In client5.1, poem is obtained asynchronously and callback function cummingsify is called for transform. Now we implement transformservice in part12, that is, poem transform also needs to be asynchronously used for the server to complete.

This is actually a natural idea. Due to the reactor's characteristics, any callback must be unblock. However, in fact, many callback processing takes a long time, at this time, the callback must also be processed asynchronously to ensure the Unblock of the callback itself. That is, the callback itself cannot directly return the result, but can only return the deferred object.

For example, when an inner deferred is encountered,

The outer deferred needs to wait until the inner deferred is fired. Of course, the outer deferredCan't blockEither, so instead the outer deferred suspends the execution of the callback chain and returns control to the reactor

AndHow does the outer deferred know when to resume? Simple-Adding a callback/errback pair to the inner deferred. Thus, when the inner deferred is fired the outer deferred will resume executing its chain. if the inner deferred succeeds (I. E ., it callthe callback added by the outer deferred), then the outer deferred callitsN + 1Callback with the result. And if the inner deferred fails (callthe errback added by the outer deferred), the outer deferred calltheN + 1Errback with the failure.

 

The following code encapsulates inner deferred to provide asynchronous callback,

class TransformClientProtocol(NetstringReceiver):     def connectionMade(self):        self.sendRequest(self.factory.xform_name, self.factory.poem)     def sendRequest(self, xform_name, poem):        self.sendString(xform_name + '.' + poem)     def stringReceived(self, s):        self.transport.loseConnection()        self.poemReceived(s)     def poemReceived(self, poem):        self.factory.handlePoem(poem)        class TransformClientFactory(ClientFactory):     protocol = TransformClientProtocol     def __init__(self, xform_name, poem):        self.xform_name = xform_name        self.poem = poem        self.deferred = defer.Deferred()     def handlePoem(self, poem):        d, self.deferred = self.deferred, None        d.callback(poem)     def clientConnectionLost(self, _, reason):        if self.deferred is not None:            d, self.deferred = self.deferred, None            d.errback(reason)     clientConnectionFailed = clientConnectionLost    class TransformProxy(object):    """    I proxy requests to a transformation service.    """     def __init__(self, host, port):        self.host = host        self.port = port     def xform(self, xform_name, poem):        factory = TransformClientFactory(xform_name, poem)        from twisted.internet import reactor        reactor.connectTCP(self.host, self.port, factory)        return factory.deferreddef cummingsify(poem):    d = proxy.xform('cummingsify', poem)     def fail(err):        print >>sys.stderr, 'Cummingsify failed!'        return poem     return d.addErrback(fail)

Finally, this function is encapsulated asynchronous callback. You can compare it with the call back of part10...

def cummingsify(poem):    print 'First callback, cummingsify'    poem = engine.cummingsify(poem)    return poemdef cummingsify_failed(err):    if err.check(GibberishError):        print 'Second errback, cummingsify_failed, use original poem'        return err.value.args[0] #return original poem    return err

Let's take a look at the callback sequence diagram in part10. At this time, cummingsify is asynchronous callback, and cummingsify_failed is added to inner deferred. When this inner deferred is fired, outer deferred will be called according to, got_poem or poem_failed. the specific process seems transparent... or I don't know.

I have not made it clear here. I personally think it would be clearer if I could refer to part10 to give a complete code example here...

Part 14: When a deferred isn' t

We'll make a Caching Proxy Server. When a client connects to the proxy, the proxy will either fetch the poem from the external server or return a cached copy of a previusly retrieved poem.

It can be seen that, if it is returned directly from the cache, it can be directly processed synchronously. If it needs to be obtained from the external server, it needs to be processed asynchronously.

What should I do if synchronization is required and asynchronous?

In the following code, get_poem may return a poem or a deferred object. How can this problem be solved by the caller...

class ProxyService(object):     poem = None # the cached poem     def __init__(self, host, port):        self.host = host        self.port = port     def get_poem(self):        if self.poem is not None:            print 'Using cached poem.'            return self.poem         print 'Fetching poem from server.'        factory = PoetryClientFactory()        factory.deferred.addCallback(self.set_poem)        from twisted.internet import reactor        reactor.connectTCP(self.host, self.port, factory)        return factory.deferred     def set_poem(self, poem):        self.poem = poem        return poemclass PoetryProxyProtocol(Protocol):     def connectionMade(self):        d = maybeDeferred(self.factory.service.get_poem)        d.addCallback(self.transport.write)        d.addBoth(lambda r: self.transport.loseConnection()) class PoetryProxyFactory(ServerFactory):     protocol = PoetryProxyProtocol     def __init__(self, service):        self.service = service

UseMaybedeferredTo solve this problem, this function will encapsulate poem into an already-fired deferred

• If the function returns a deferred,maybeDeferredReturns that same deferred, or
• If the function returns a failure,maybeDeferredReturns a new deferred that has been fired (.errback) With that failure, or
• If the function returns a regular value,maybeDeferredReturns a deferred that has already been fired with that value as the result, or
• If the function raises an exception,maybeDeferredReturns a deferred that has already been fired (.errback()) With that exception wrapped in a failure.

An already-fired deferred may fire the new callback (or errback, depending on the state of the deferred)Immediately, I. e., right when you add it.

Or useSucceedFunction,defer.succeedFunction is just a handy way to make an already-fired deferred given a result.

 

def get_poem(self):    if self.poem is not None:        print 'Using cached poem.'        # return an already-fired deferred        return succeed(self.poem)     print 'Fetching poem from server.'    factory = PoetryClientFactory()    factory.deferred.addCallback(self.set_poem)    from twisted.internet import reactor    reactor.connectTCP(self.host, self.port, factory)    return factory.deferred