Synchronous Programming Model of asynchronous operations based on python yield mechanism, pythonyield

Synchronous Programming Model of asynchronous operations based on python yield mechanism, pythonyield

This article summarizes how to synchronize and simulate asynchronous operations when writing python code to improve code readability and scalability.

Generally, the game engine uses a distributed framework to balance the resource loads of Server clusters through certain policies, so as to ensure high concurrency and high CPU utilization of server operations, and ultimately improve the Game Performance and load. Because the logic layer calls of the engine are non-preemptible and communication between servers is performed asynchronously, the game logic cannot be executed synchronously. Therefore, many callback functions must be manually added to the Code layer, distributes a complete function in various callback functions in a fragmented manner.

Asynchronous Logic

Take the replica scoring logic in the game as an example. At the end of the replica, the replica management process needs to collect the combat information of each player in the replica, and finally provide a replica score based on the statistics in the management process, reward. Because each player entity is randomly distributed in different processes, the management process needs to obtain the combat information of the player through asynchronous calls.

The implementation code is as follows:

#-*-Coding: gbk-*-import random # Player entity class Player (object): def _ init _ (self, entityId): super (Player, self ). _ init _ () # player ID self. entityId = entityId def onFubenEnd (self, mailBox): score = random. randint (1, 10) print "onFubenEnd player % d score % d" % (self. entityId, score) # Send your id and Combat Information mailBox to the replica management process. onEvalFubenScore (self. entityId, score) # copy management class FubenStub (object): def _ init _ (self, players): super (FubenStub, self ). _ init _ () self. players = players def evalFubenScore (self): self. playerRelayCnt = 0 self. totalScore = 0 # notify each registered player that the copy has ended and the combat information is requested for player in self. players: player. onFubenEnd (self) def onEvalFubenScore (self, entityId, score): # receives the Combat Information of one of the players print "onEvalFubenScore player % d score % d" % (entityId, score) self. playerRelayCnt + = 1 self. totalScore + = score # if len (self. players) = self. playerRelayCnt: print 'the fuben totalScore is % d' % self. totalScore if _ name _ = '_ main _': # simulate Player entity creation players = [Player (I) for I in xrange (3)] # When a copy starts, each player registers its own MailBox to the copy management process fs = FubenStub (players) # The copy is in progress #.... # When the copy ends, the score starts to fs. evalFubenScore ()

The Code simplifies the implementation of the replica scoring logic. The Player class represents the Player entity of the game, seamlessly switches between different servers during game operation, and FubenStub represents the replica management process, at the beginning of the copy, all players in the copy will register their MailBox in the management process. MailBox indicates the remote call handle of each entity. At the end of the copy, FubenStub first sends the copy End message to each player and requests the player's combat information. After receiving the message, the player sends the combat information to FubenStub; then, after FubenStub collects information about all players, the score is printed.

Synchronization Logic

If Player and FubenStub are in the same process, all operations can be synchronized. When FubenStub sends a copy of the End message to the Player, it can immediately obtain the combat information of the Player, the implementation code is as follows:

# -*- coding: gbk -*- import random class Player(object):  def __init__(self, entityId):    super(Player, self).__init__()    self.entityId = entityId   def onFubenEnd(self, mailBox):    score = random.randint(1, 10)    print "onFubenEnd player %d score %d"%(self.entityId, score)    return self.entityId, score class FubenStub(object):  def __init__(self, players):    super(FubenStub, self).__init__()    self.players = players   def evalFubenScore(self):    totalScore = 0    for player in self.players:      entityId, score = player.onFubenEnd(self)      print "onEvalFubenScore player %d score %d"%(entityId, score)      totalScore += score     print 'The fuben totalScore is %d'%totalScore if __name__ == '__main__':  players = [Player(i) for i in xrange(3)]   fs = FubenStub(players)  fs.evalFubenScore()

From the above two pieces of code, we can see that due to asynchronous operations, the scoring logic in FubenStub is artificially divided into two functional points: 1) send a copy to the player to end the message; 2) Accept the player's combat information; and the two function points are distributed in two different functions. If the game logic is complex, it will inevitably lead to scattered functions and excessive onXXX asynchronous callback functions, resulting in improved code development and maintenance costs and reduced readability and scalability.

If there is a way to temporarily suspend the function during asynchronous calls and resume execution after the callback function returns the returned value, you can use synchronous programming mode to develop asynchronous logic.

Yield keywords

Yield is a keyword in Python. If the yield keyword appears in the function body, Python will change the context of the entire function. Calling this function will not return a value, but a generator object. The generator object can be executed only when the next iteration function of the generator is called. When the generator object is executed with the yield expression, the function will be suspended temporarily and wait for the next call to resume execution, the specific mechanism is as follows:

1) call the next method of the generator object to start function execution;

2) When the generator object is executed to contain the yield expression, the function is suspended;

3) The next function call will drive the generator object to continue executing subsequent statements until the next yield is suspended again;

4) if a next call drives the generator to continue execution, and then the function ends normally, the generator will throw a StopIteration exception;

The following code is used:

def f():  print "Before first yield"  yield 1  print "Before second yield"  yield 2  print "After second yield" g = f()print "Before first next"g.next()print "Before second next"g.next()print "Before third yield"g.next()

The execution result is:

Before first next

Before first yield

Before second next

Before second yield

Before third yield

After second yield

StopIteration

Ha, with the Mechanism to temporarily suspend the function, the question of how to pass the return value of asynchronous calls is left. In fact, the next function of the generator has implemented a mechanism for passing parameters from inside the generator object, and python also provides a mechanism for sending parameters from inside the external generator object, the specific mechanism is as follows:

1) when the next function is called to drive the generator, next will wait for the next yield suspension in the generator and return the parameters following the yield to next;

2) When passing parameters to the generator, you must replace the next function with the send function. In this case, the send function is the same as the next function (the driver generator executes and waits for the return value ), at the same time, send will pass the following parameters to the yield suspended before the generator;

The following code is used:

def f():  msg = yield 'first yield msg'  print "generator inner receive:", msg  msg = yield 'second yield msg'  print "generator inner receive:", msg g = f()msg = g.next()print "generator outer receive:", msgmsg = g.send('first send msg')print "generator outer receive:", msgg.send('second send msg')

The execution result is:

Generator outer receive: first yield msg

Generator inner receive: first send msg

Generator outer receive: second yield msg

Generator inner receive: second send msg

StopIteration

Synchronous implementation

Well, everything is ready for development. The following is a simple project encapsulation of the yield mechanism for future development. The following code provides an interface named IFakeSyncCall. All logical classes containing asynchronous operations can inherit this interface:

class IFakeSyncCall(object):  def __init__(self):    super(IFakeSyncCall, self).__init__()    self.generators = {}   @staticmethod  def FAKE_SYNCALL():    def fwrap(method):      def fakeSyncCall(instance, *args, **kwargs):        instance.generators[method.__name__] = method(instance, *args, **kwargs)        func, args = instance.generators[method.__name__].next()        func(*args)      return fakeSyncCall    return fwrap   def onFakeSyncCall(self, identify, result):    try:      func, args = self.generators[identify].send(result)      func(*args)    except StopIteration:      self.generators.pop(identify)

The generators attribute in the interface is used to save the generator objects that have started execution in the class. The FAKE_SYNCALL function is a decorator. The decoration class contains the yield function and changes the call context of the function, the next call to the generator object is encapsulated in fakeSyncCall; The onFakeSyncCall function encapsulates the logic of all onXXX functions, and other entities pass the asynchronous callback return value by calling this function.

The following is the logic code of asynchronous copy scoring after synchronization improvement:

# -*- coding: gbk -*-import random class Player(object):  def __init__(self, entityId):    super(Player, self).__init__()    self.entityId = entityId   def onFubenEnd(self, mailBox):    score = random.randint(1, 10)    print "onFubenEnd player %d score %d"%(self.entityId, score)    mailBox.onFakeSyncCall('evalFubenScore', (self.entityId, score)) class FubenStub(IFakeSyncCall):  def __init__(self, players):    super(FubenStub, self).__init__()    self.players = players   @IFakeSyncCall.FAKE_SYNCALL()  def evalFubenScore(self):    totalScore = 0    for player in self.players:      entityId, score = yield (player.onFubenEnd, (self,))      print "onEvalFubenScore player %d score %d"%(entityId, score)      totalScore += score     print 'the totalScore is %d'%totalScore if __name__ == '__main__':  players = [Player(i) for i in xrange(3)]   fs = FubenStub(players)  fs.evalFubenScore()

Compared with the evalFubenScore function, it is almost the same as the original synchronization logic code.

Another advantage of implementing the Synchronous Programming Model Using the yield mechanism is that it can ensure the logic serialization of all asynchronous calls, thus ensuring data consistency and effectiveness, in particular, the traditional timer sleep mechanism can be abandoned in various asynchronous initialization processes to eliminate some hidden bugs caused by data inconsistency from the source.

Articles you may be interested in:

• Python in-depth understanding of yield
• The tool for automatic synchronization of computer time written in Python
• Python yield usage example
• How to use yield in python
• Python asynchronous task queue example
• Test the performance of asynchronous Socket programming in Python
• Python multi-thread synchronization Lock, RLock, Semaphore, Event instance
• Generator and yield in Python
• Simplify complex programming models with Python SimPy Library