Detailed game server architecture upgrade record [dry goods]

I. Agricultural age

The most important thing about entrepreneurship is the word "fast". Therefore, at the beginning, all architectures were created on the premise of a fast model.

My blog friends often know that I have a special liking for python. Naturally, python has become the language for developing the server framework.

The framework of the multi-threaded tcp server provided by python is very simple: ThreadingTCPServer, that is, the mode in which each link is a thread:

Import SocketServer

Class RequestHandler (SocketServer. BaseRequestHandler ):
Def handle (self ):
F = self. request. makefile ('r ')

While True:
Message = f. readline ()
If not message:
Print 'Client closed'
Break
Print "message, len: % s, content: % r" % (len (message), message)
Self. request. send ('okn ')


Class MyServer (SocketServer. ThreadingTCPServer ):
Request_queue_size = 256
Daemon_threads = True
Allow_reuse_address = True


Server = MyServer ('127. 0.0.1 ', 127), RequestHandler)
Server. serve_forever ()

However, I think the use of multi-threaded locking is too troublesome, so gevent is introduced:

Import gevent
From gevent. server import StreamServer

Class RequestHandler (object ):

Closed = False

Def _ init _ (self, sock, address ):
Self. sock = sock
Self. address = address
Self. f = self. sock. makefile ('r ')
Self. handle ()

Def handle (self ):
While not self. closed:
T = gevent. spawn (self. read_message)
T. join ()

Def read_message (self ):
Message = self. f. readline ()
If not message:
Self. closed = True
Print 'Client closed'
Return
Print "message, len: % s, content: % r" % (len (message), message)
Self. sock. send ('okn ')


Server = StreamServer ('127. 0.0.1 ', 127), RequestHandler)
Server. serve_forever ()

As I used to be a personal developer, I liked flask's decorator design very much. So I designed my own tcp server by referring to flask:

Https://github.com/dantezhu/haven

The usage is also very simple (on the server side ):

Import logging

From haven import GHaven, THaven, logger
From netkit. box import Box

App = GHaven (Box)

@ App. before_request
Def before_request (request ):
Logger. error ('before _ request ')

@ App. route (1)
Def index (request ):
Request. write (dict (ret = 100 ))

App. run ('127. 0.0.1 ', 127, workers = 2)

Client:

From netkit. contrib. tcp_client import TcpClient
From netkit. box import Box

Import time

Client = TcpClient (Box, '127. 0.0.1 ', 127, timeout = 5)
Client. connect ()

Box = Box ()
Box. cmd = 101
Box. body = 'I love you'

Client. write (box)

T1 = time. time ()

While True:
# Blocking
Box = client. read ()
Print 'time past: ', time. time ()-t1
Print box
If not box:
Print 'server closed'
Break

This architecture is very effective when developing server-side prototypes because of its high development efficiency.

We developed a card and board game. At that time, for the convenience of graphs, almost all the data was stored in the process memory, so the operation was very convenient.

Therefore, during the entire development process, the development speed on the server side has been several times faster than that on the client side.

II. Industrial age

However, soon I found some problems with the existing framework, which were extremely fatal.

1. All logic is in one process, and the performance is too low. If 2000 people play cards at the same time, it will lead to choppy

2. Multi-threaded models provide poor performance when a large number of users are online

3. The logic server and storage server are joined together, making maintenance very difficult. Restarting the logical server will affect your business and will be unacceptable.

For the above reasons, I have been considering a new business model. In addition to solving the above problems, it also has the following features:

1. Keep the python development business logic as much as possible, because the development efficiency is extremely high compared with c ++.

2. Scalable and distributed

3. Try to change the existing logic code as little as possible

4. Try to make business development easy to understand

 

Finally, I implemented this server framework and made it open-source here:

Https://github.com/dantezhu/maple

The implementation of maple has been inspired by many ideas and frameworks, including zmq, half-async half-sync, and a business model shared by Taobao at that time.

I remember clearly that the speaker said this:

How can we determine whether to send messages to a server?

Depends on the success rate? Based on the response time?

No. We replace push with pull. After the worker completes the processing, it will ask for data by itself.

Yes, I will give it. If not, I will not give it.

 

This is true for the entire maple model:

Gateway
Worker
Trigger

The gateway is a high-performance forwarding server implemented by c ++ and epoll. All received client messages are forwarded to the corresponding worker.

A worker, that is, a worker process, can attach data to a gateway at any time or detach data at any time. In addition, worker uses python to achieve both development efficiency and operation efficiency.

Trigger, that is, the trigger that can trigger the event to send to the gateway. The gateway sends the event to the client or worer based on the event.

 

For detailed design ideas, refer to the readme of maple, which contains detailed design ideas.

A simple worker code is as follows:

Import logging

LOG_FORMAT = 'N'. join ((
'/' + '-' * 80,
'[% (Levelname) s] [% (asctime) s] [% (process) d: % (thread) d] [% (filename) s: % (lineno) d % (funcName) s]: ',
'% (Message) s ',
'-' * 80 + '/',
))

Logger = logging. getLogger ('Maple ')
Handler = logging. StreamHandler ()
Handler. setFormatter (logging. Formatter (LOG_FORMAT ))
Logger. addHandler (handler)
Logger. setLevel (logging. DEBUG)


From maple import Worker
From netkit. box import Box

App = Worker (Box)

@ App. close_client
Def close_client (request ):
Logger. error ('close _ client: % R', request)

@ App. route (1)
Def test (request ):
Request. write_to_client (dict (
Ret = 0,
Body = "test"
))

App. run ("192.168.1.67", 28000, workers = 2, debug = True)

A simple trigger code is as follows:

From maple import Trigger
Import time
From netkit. box import Box

Import logging

Logger = logging. getLogger ('Maple ')
Logger. addHandler (logging. StreamHandler ())
Logger. setLevel (logging. DEBUG)


Def main ():
Trigger = Trigger (Box, '192. 168.1.67 ', 192)
# Trigger = Trigger (Box, '2017. 28.224.64 ', 115)

For it in xrange (0, 99999 ):
Time. sleep (1)

Print trigger. write_to_worker (dict (
Cmd = 3,
Ret = 100,
Body = 'from trigger: % s' % it
))
# Print trigger. close_users ([-1, 3])
# Print trigger. write_to_users ([
# (1, 2, 3), dict (cmd = 1, body = 'direct event from trigger: % s' % it ))
#])


Main ()

Very simple, right?

It is worth mentioning that, in order to make it easier for worker to restart at any time without affecting Internet services, worker implements special processing for USR1 and USR2 signals. Stop all processes and pull workers again.

 

Finally, the command is as follows:

./Tool_stat-f stat_file


Github link:

Https://github.com/dantezhu/haven

Https://github.com/dantezhu/maple