Tao Analysis of GOLANG-TCP asynchronous frame

This is a creation in Article, where the information may have evolved or changed.

TCP Asynchronous framework

Golang Programming Style

• Go language object-oriented programming style is a multi-use combination, less inheritance , in the way of anonymous embedding to implement inheritance.

Master the Go language, to grasp a center, two basic points.

• a center is the Go language concurrency model, that is, do not communicate through shared memory, to share memory through communication ;
• The two basic points are the two cornerstones of the Go language concurrency model: channel and go-routine .

Do not use shared memory to communicate, to share memory through communication

The general explanation of this sentence is: do not use shared memory to achieve communication, this is because of complex distributed, multi-threaded and multi-process through the locking and other control concurrency to ensure the correctness of the data, is very difficult and inefficient. It is suggested that the channel should be communicated between threads channel to reduce the competition of data and improve the reliability and correctness of the system.

1. Start of service start

1.1 Start the heartbeat timer loop

 func (S *server) Timeoutloop () {defer s.wg.done () for {select {case <-s.ct X.done (): return case Timeout: = <-s.timing.timeoutchannel (): NetID: = timeout. Ctx.value (NETIDCTX). (Int64) if V, OK: = S.conns.load (NetID); OK {sc: = V. (*serverconn) Sc.timerch <-Timeout} else {Holme S.WARNF ("Invalid client%d", NetID)}}}  

When the service starts, a timer loop is timeOutLoop used to maintain Clinet the application- level heartbeat of the connection service, and in one goroutine , through the select monitoring service is called the timeOutChan scheduled task channel ,
If there is a timed task coming, get through the context context netIDCtx , this is the TCP connection unique ID, according to this ID we can find the correspondingServerConn
(Serverconn: This is for a TCP connection, the upper-level connection encapsulation.) It consists of three important channel , respectively, sendCh handlerCh and timerCh , as detailed below ).

This allows the scheduled expiration task to be placed in the corresponding ServerConn timeCh , and the connection handles the execution of the scheduled expiration task.

1.2 Service start limit handling

• If a temporary error occurs when the server accepts a client connection request, the server waits up to 1 seconds before attempting to accept the request again.
• If the number of existing connections exceeds MaxConnections (default 1000), the connection is rejected and closed, otherwise a new connection starts working.

2. Network Connection Processing module

func (sc *ServerConn) Start() {    holmes.Infof("conn start, <%v -> %v>\n", sc.rawConn.LocalAddr(), sc.rawConn.RemoteAddr())    onConnect := sc.belong.opts.onConnect    if onConnect != nil {        onConnect(sc)    }    loopers := []func(WriteCloser, *sync.WaitGroup){readLoop, writeLoop, handleLoop}    for _, l := range loopers {        looper := l        sc.wg.Add(1)        go looper(sc, sc.wg)    }}

In other programming languages, servers that are Reactor written in patterns often need to be multiplexed asynchronously through an IO thread epoll . And because the cost of the go thread is cheap, the go language can create three goroutine for each network connection.

• readLoop()is responsible for reading the data and deserializing it into a message.
• writeLoop()is responsible for serializing the message and sending a binary stream of bytes.
• handleLoop()is responsible for invoking the message handler function.

These three processes run independently when the connection is created and started.

2.1 Readloop Implementation Details

    For {select {case <-cdone://Connection closed Holmes. Debugln ("Receiving cancel signal from Conn") return case <-sdone://server closed Holmes. Debugln ("Receiving cancel signal from server") return default:msg, err = codec. Decode (rawconn) if err! = Nil {Holmes. Errorf ("Error decoding message%v\n", err) If _, OK: = Err. (errundefined); OK {//update heart beats Setheartbeatfunc (time. Now (). Unixnano ()) Continue} return} Setheartbeatfunc (Ti Me. Now (). Unixnano ()) Handler: = Gethandlerfunc (Msg. MessageNumber ()) If Handler = = nil {if onMessage! = Nil {Holmes. Infof ("Message%d call OnMessage () \ n", Msg.   MessageNumber ()) OnMessage (MSG, C. (Writecloser))} else {                 Holmes. WARNF ("No handler or onMessage () found for message%d\n", MSG. MessageNumber ())} continue} Handlerch <-messagehandler{msg, hand Ler}}}

The packets in the network connection are read through in the Readloop loop codec rawConn , and the parsed data is returned.
The codec parsing function used is registered when the service is started, and the execution function for that type of data is registered, and the message type is key saved in the message.go package.
After the parse succeeds, the execution function of the message is obtained, and the two packages are MessageHandler sent to handlerCh medium. For HandleLoop loop execution.

2.2 Handleloop Implementation Details

For {select {case <-cdone://Connectin closed Holmes. Debugln ("Receiving cancel signal from Conn") return case <-sdone://server closed Holmes. Debugln ("Receiving cancel signal from server") return case Msghandler: = <-handlerch:msg, Handler: = Msghandler.message, Msghandler.handler if handler! = Nil {if Askforworker {err = Worker Poolinstance ().                Put (NetID, func () {handler (Newcontextwithnetid (Newcontextwithmessage (CTX, msg), NetID), c) }) If Err! = Nil {Holmes. ERRORLN (Err)} addtotalhandle ()} else {handler (newcontextwithnet  ID (Newcontextwithmessage (CTX, msg), NetID), c)}} Case Timeout: = <-timerch:if Timeout! = Nil {timeoutnetid: = Netidfromcontext (timeout. CTX) if Timeoutnetid! = NetID {Holmes. ErrorF ("Timeout net%d, conn net%d, mismatched!\n", Timeoutnetid, NetID)} if Askforworker { Err = Workerpoolinstance (). Put (NetID, func () {timeout. Callback (time. Now (), C. (Writecloser))}) if Err! = Nil {Holmes. ERRORLN (ERR)}} else {timeout. Callback (time. Now (), C. (Writecloser))}}}}

In the HandleLoop loop, the main listener handlerCh and timerCh , one is the message execution channel , one is the timed task expires channel .

• handlerChProcessing is the data sent over the Readloop loop, and the task is performed through an asynchronous task pool.
• timerChHandles the execution of timed tasks under this connection, and also performs tasks through an asynchronous task pool.

2.2 Writeloop Implementation Details

for {    select {    case <-cDone: // connection closed        holmes.Debugln("receiving cancel signal from conn")        return    case <-sDone: // server closed        holmes.Debugln("receiving cancel signal from server")        return    case pkt = <-sendCh:        if pkt != nil {            if _, err = rawConn.Write(pkt); err != nil {                holmes.Errorf("error writing data %v\n", err)                return            }        }    }}

func ProcessMessage(ctx context.Context, conn tao.WriteCloser) {    msg := tao.MessageFromContext(ctx).(Message)    holmes.Infof("receving message %s\n", msg.Content)    conn.Write(msg)}

In the WriteLoop loop, the main listener sendCh , it will be non-blocking the message in the SENDCH all sent and then quit, to avoid missing messages.
The sendCh incoming message is registered at the beginning of the service message , and is ProcessMessage Write written to it asynchronously sendCh .

3. Summary

In the Tao framework of the three loops ReadLoop , HandleLoop and is the WriteLoop entire core code, these three Loop are through channel to achieve data transfer, and each TCP connection will achieve these three goroutine . Each one goroutine is run independently.

• The framework is supported by Tao. The Tlscredsoption () function provides transport Layer security for TLS Server

• And in our development of different businesses, writing business code is in the customization message . You need to implement DeserializeMessage a function that parses the type's packet function and ProcessMessage uses the message only.

• The use of the contact program in the framework of the context text, so that the program can gracefully exit.

• ContextThe use of Golang concurrency model in another article.

• As for the implementation analysis of timers in the framework in another article golang-based on Timeingwheel timers.

4. Thanks

Thanks leesper for contributing to the open source community and providing us with learning.

Tao Source