Gfirefly Framework Analysis

Data packageAgreement

The agreement is the following table
| Head_0 | ... | Head_3 | protoversion | ServerVersion | Length |
|--–|--–|--–|--–|--–|--–|
| char | ... | char | int | int | int |
Gfirefly Framework is a long connection based on the TCP protocol, the framework does not use the keep-alive, then the network abnormal disconnection (such as the sudden removal of cable), the application layer is not known, when we use the time we have to add a heartbeat package and other mechanisms to solve the problem. In addition, data encryption is not placed in the protocol layer, so the need to encrypt data can only be used before packaging the data encryption algorithm.

Sticky bag

The underlying packet is gfirefly, as shown in the table above, and the protocol header defines length, at which point the application layer knows when to end the transmission of a single packet. See gfirefly/netconnect/protoc.py Line 43:

def dataReceived(self, data):    length = self.factory.dataprotocl.getHeadlength()#获取协议头的长度    self.buff += data    while self.buff.__len__() >= length:        unpackdata = self.factory.dataprotocl.unpack(self.buff[:length])        if not unpackdata.get(‘result‘):            log.msg(‘illegal data package --‘)            self.factory.connmanager.loseConnection(self.transport.sessionno)            break        command = unpackdata.get(‘command‘)        rlength = unpackdata.get(‘length‘)        request = self.buff[length:length+rlength]        if request.__len__()< rlength:            log.msg(‘some data lose‘)            break

Internet

Gfirefly is modified by the Firefly, then when the underlying network module changed from twisted to Gevent, added a gtwisted called wrapper. The gevent is actually used. Streamserver. There are still some uncomfortable places, and has not been modified, such as gtwisted/core/base.py Line 122 or so, GetHost function manually return "0.0.0.0" or something.

Rpc

| Length | Data |
|--–|--–|
| int | string |
RPC protocol header such as the table above, where data is dict by Marshal results, Marshal Speed can be, I test Marshal and msgpack speed, but Msgpack can cross platform, hehe. The inside of data is this: it {‘_msgtype‘: _msgtype, ‘_key‘: _key, ‘_name‘: _name, ‘_args‘: args, ‘_kw‘: kw} looks nothing special.

Distributed

The two-way invocation of RPC is implemented in Gfirefly, making it easier to implement the distribution. But I test the efficiency of two-way RPC is very low, the QPS is about 4000, and test one-way Mprpc QPS reached 1w8.

1. The gate node is the central node.
2. There are child nodes connected to the Gate,child to communicate with each other also through gate nodes to relay.

Cache

The framework encapsulates a module that uses memcache to cache MySQL data, which is limited to tables with primary keys or foreign keys, but is sufficient. The disadvantage is that value holds a large dictionary, and each time a call is modified, the entire value of the store needs to be modified to be less efficient. If the hset of Redis would be much better, I would test the Redis drive and the Memcache driver myself in contrast. The results are as follows

| Redis-py | Credis | Memcache | PYLIBMC |
|--–|--–|--–|
| 2w+ | 4w+ | 2w+ | 5w+ |
Add: Credis does not encapsulate a function on the command.

The pit in the Gevent

When simulating a large number of logins, the program slows down. The official MySQL uses Mysql-python (the bottom layer is C/s + +), is very fast in all MySQL drivers, slower than umysql, but umysql incompatible DB-API 2.0. During the test, Gevent is blocked when the query is called inside. After the greenify with watercress did not succeed, and eventually replaced by Pymysql. Memcache driver replaced with pylibmc,memcache fast enough, if the gevent will only slow down the speed, the results of the search is gevent there will be a lot of system calls, = = sub-millisecond is not recommended greenlet==.

All things belong to one

Use scenario Analysis: This framework is only suitable for some simple (game) server applications. Python is very cool, but speed is a big problem, even using a variety of black technology, may also be unsatisfactory ah, after all, many game servers are both I/O and CPU-intensive.

Gfirefly Framework Analysis