Using the python-twisted Library to implement a file transceiver service

Recently in the Learning Python-twisted Library, before doing asynchronous concurrent programming has always been using C + +, such as the epoll mechanism under Linux, Windows IOCP mechanism, to the later used Boost::asio library, compared to the C + + The async experience on the twisted is a bit of a hindrance to my learning the library for the following reasons:

Both Epoll and IOCP are given a notification when a read operation or write operation is performed (IOCP is complete), and then you can execute your next code. In this scenario, if you are sending and receiving large data, you can continue to send or accept the next block of data when you are notified of the necessary processing, but the twisted library is not designed like this. In Twisted.protocols.basic.LineReceiver, for example, when data is received asynchronously, the class's linereceived function or rawdatareceived (depending on the current schema) is triggered, but it does not contain an asynchronous write function, which means you cannot do this For

1. Call the asynchronous send function to send the data

2. Trigger a callback function when data send is complete

3, in the callback function you decide to follow the operation, such as continue to send the next piece of data

So what's the solution for twisted? The answer is Producer/consumer, producer is responsible for the continuous generation of data (such as constantly reading from file), consumer is responsible for processing data (through the socket sent out), When consumer is fully cached when it is sent, it notifies producer to pause providing data, and when the cache is emptied, continue to notify producer to generate the data (which is similar to what we mentioned above, when a callback function is called when the asynchronous send completes), this procedure does not require the control that we display in the code. Due to the universality of the file sending application, twisted provides a file sending class Filesender, OK, descriptive content is here, follow-up about Producer/consumer I will further introduce, the following directly on the code

 fromTwisted.internet.protocolImportprotocol,factory fromTwisted.protocols.basicImportLinereceiver fromTwisted.protocols.basicImportFilesender fromTwisted.internet.deferImportDeferredImportpickle,structImportMultiprocessing.poolImportOs,jsonclassTransferfileprotocol (linereceiver): Port_=0def __init__(self): self.handle_f_=None#--------------------------Self.instruction_ =None Self.command_=None Self.size_remain_=0#Self.setrawmode ()    def_monitor (self,data): Self.size_remain_-=len (data)returnDatadefcbtransferover (Self, lastsent):Print('Download over!') self.transport.loseConnection ()defConnectionmade (self):Print('Got Connection from', Self.transport.client)Print 'Work on port:%d'%(Self.port_,)defConnectionlost (Self,reason):Print(Self.transport.client,'disconnected!')        ifSelf.handle_f_! =None:self.handle_f_.close ()ifSelf.size_remain_! =0:Print 'Transfer File fail!'#if Self.command_ = = ' put ':#OS.    deflinereceived (self, line): Self.instruction_=json.loads (line) Self.command_= self.instruction_['Command'] Name_file= self.instruction_['Name_file']        ifSelf.command_ = ='put': Self.size_remain_= self.instruction_['Size_file']            Try: Self.handle_f_= Open ('files/%s'% (Name_file,),'WB')            except:                Print 'Open%s fail!'%(Name_file,) self.handle_f_=None#self.transfer.loseConnection ()            Else: Self.setrawmode ()elif 'Get'==Self.command_: Self.size_remain_=0ifOs.path.exists ('files/%s'%(Name_file,)): Self.size_remain_= Os.stat ('files/%s'%(Name_file,)). st_size instruction= Dict (size_file=self.size_remain_)ifSelf.size_remain_>0:Try: Self.handle_f_= Open ('files/%s'% (Name_file,),'RB')                except: Self.size_remain_=0 self.handle_f_=NoneElse: Self.setrawmode () self.transport.write (json.dumps (instruction)+'\ r \ n')            ifSelf.size_remain_>0:sender=Filesender () sender. Chunck_size= 2**16D=Sender.beginfiletransfer (self.handle_f_,self.transport,self._monitor) d.addcallback (self.cbTransferOve R)defrawdatareceived (self, data):Print 'Length of data:%d'%(len (data),)if 'put'= = Self.command_ andSelf.handle_f_! =None:self.handle_f_.write (data) Self.size_remain_-=len (data)defRecv_func (port): fromTwisted.internetImportepollreactor Epollreactor.install () fromTwisted.internetImportreactor Transferfileprotocol.port_=Port Factory=Factory () Factory.protocol=transferfileprotocol reactor.listentcp (port,factory) reactor.run ()if __name__=='__main__': Ports= [6200,6202,6204,6206] Pool=Multiprocessing.pool.Pool (len (ports)) Pool.map (recv_func,ports)#Recv_func (6200)

Using the python-twisted Library to implement a file transceiver service