The implementation of the Fasthttp pool in the process

This is a creation in Article, where the information may have evolved or changed.

The implementation of the Fasthttp pool in the process

The pool can control the degree of parallelism and reuse the coprocessor. The important reason that fasthttp is much more efficient than net/http is that it utilizes the co-process pool. The implementation is not complicated, we can refer to his design, write high-performance applications.

Entrance

Server.gofunc (S *server) Serve (ln net. Listener) Error {var lastoverflowerrortime time. Time Var lastperiperrortime time. Time var c net.    Conn var err error Maxworkerscount: = s.getconcurrency () S.concurrencych = Make (chan struct{}, Maxworkerscount) WP: = &workerpool{workerfunc:s.serveconn, Maxworkerscount:maxworkerscount, logallerrors : S.logallerrors, Logger:s.logger (),}//break-01 WP.            Start () for {//break-02 if c, err = Acceptconn (s, LN, &lastperiperrortime); Err! = Nil { Wp. Stop () If err = = Io. EOF {return nil} return err}//break-03 if!wp. Serve (c) {S.writefasterror (c, statusserviceunavailable, "the connection cannot be served Becaus E server.concurrency limit Exceeded ") c.close () if time. Since (Lastoverflowerrortime) > time.            Minute {    S.logger (). PRINTF ("The incoming connection cannot be served, because%d concurrent connections is served.") + "Try increasing server.concurrency", maxworkerscount) Lastoverflowerrortime = Coarseti Menow ()}//The current server reached concurrency limit,//so give other concurrently            Running servers a chance//accepting incoming connections on the same address. There is a hope other servers didn ' t reach their//concurrency limits yet:) time. Sleep (* time. Millisecond)} c = Nil}}//It is necessary to understand the structure of the Workerpool type workerpool struct {//Function for serving Ser    ver connections.    It must leave C unclosed. Workerfunc func (c net. Conn) Error Maxworkerscount int logallerrors bool maxidleworkerduration time. Duration Logger Logger lock sync. Mutex Workerscount int muststop bool Ready []*workerchan STopch Chan struct{} workerchanpool sync. Pool}

Goroutine Status:

1. Main0:wp. Start ()

Break-01

// workerpool.go// 启动一个 goroutine， 每隔一段时间，清理一下 []*workerChan; // wp.clean() 的操作是 查看最近使用的workerChan, 如果他的最近使用间隔大于某个值，那么把这个workerChan清理了。func (wp *workerPool) Start() {    if wp.stopCh != nil {        panic("BUG: workerPool already started")    }    wp.stopCh = make(chan struct{})    stopCh := wp.stopCh    go func() {        var scratch []*workerChan        for {            wp.clean(&scratch)            select {            case <-stopCh:                return            default:                time.Sleep(wp.getMaxIdleWorkerDuration())            }        }    }()}

Goroutine Status:

1. Main0:wp. Start ()

2. G1:for Loop to clean idle Workerchan

break-02

acceptConn(s, ln, &lastPerIPErrorTime)Mainly deals with Ln. Accept () to determine if err is temporary and eventually returns a net. Conn

break-03

// workerpool.gofunc (wp *workerPool) Serve(c net.Conn) bool {    // break-04    ch := wp.getCh()    if ch == nil {        return false    }    ch.ch <- c    return true}type workerChan struct {    lastUseTime time.Time    ch          chan net.Conn}

Wp.getch () returns a *workerchan that can be seen, Workerchan has an ch attribute, and the parameter is passed in the net. Conn directly into the inside of the plug.

break-04

Workerpool.gofunc (WP *workerpool) getch () *workerchan {var ch *workerchan createworker: = False Wp.lock.Lock () Ready: = Wp.ready N: = Len (Ready)-1 if n < 0 {//Ready is empty and the total is less than maxworkerscount, then a new WORKERC is required to be created Han if Wp.workerscount < WP. Maxworkerscount {Createworker = True wp.workerscount++}} else {//Remove a wor from the tail of the queue        Kerchan ch = ready[n] ready[n] = nil Wp.ready = ready[:n]} wp.lock.Unlock () if ch = = Nil { If!createworker {return nil}//go into the creation process and remove Workerchan VCH from the pool: = Wp.workerchan Pool.get () if vch = = Nil {vch = &workerchan{Ch:make (chan net. Conn, Workerchancap),}} ch = vch. (*workerchan)//Create Goroutine processing request, receive a Chan *workerchan as parameter Go func () {//break-05 WP . Workerfunc (CH) wp.workerChanPool.Put (VCH)} ()}    Return CH} 

Here we look only at the process of creation. If ready is empty, it means that ready is exhausted and less than Maxworkerscount, then a new Workerchan needs to be created.
When created, first remove the reuse from the Pool, and if nil, create a new.
It can be predicted that here Wp.workerfunc (CH) must contain a for loop to process the net in Workerchan. Conn.

Goroutine Status:

1. Main0:wp. Start ()

2. G1:for Loop to clean idle Workerchan

3. G2:wp.workerFunc (CH) blocks for handling connection

break-05

 //Workerpool.gofunc (WP *workerpool) workerfunc (ch *workerchan) {var c net.        Conn var err error for C = Range ch.ch {if c = = nil {break}//True processing request function If Err = WP. Workerfunc (c); Err! = Nil && Err! = errhijacked {errstr: = Err. Error () if WP. logallerrors | | ! (Strings.                Contains (ERRSTR, "broken Pipe") | | Strings.                Contains (ERRSTR, "reset by peer") | | Strings. Contains (ERRSTR, "I/O timeout")) {WP.         logger.printf ("error when serving connection%q<->%q:%s", C.localaddr (), c.remoteaddr (), Err)}}        If err! = errhijacked {c.close ()} c = nil//release Workerchan//break-06 If!wp.release (CH) {Break}}//jumps out of the for Range loop, which means to get a nil from Chan, or wp.muststop is set to True    , this is the active stop method. Wp.lock.Lock () wp.workerscount--wp.lock.Unlock ()}  

The for range continues from Chan net. Gets the connection in the Conn. Do you remember that in the func (wp *workerPool) Serve(c net.Conn) bool function, an important operation is to put the accept to the connection, into the channel.
Finally, the current Workerchan needs to be released back into Workerpool's ready.

break-06

func (wp *workerPool) release(ch *workerChan) bool {    ch.lastUseTime = CoarseTimeNow()    wp.lock.Lock()    if wp.mustStop {        wp.lock.Unlock()        return false    }    wp.ready = append(wp.ready, ch)    wp.lock.Unlock()    return true}

During the release operation, notice that the ch.lastusetime has been modified, do you remember the clean operation? Running in the G1 process.
So the final operating state is:

Goroutine Status:

1. Main0:wp. Start ()

2. G1:for Loop to clean idle Workerchan

3. G2:wp.workerFunc (CH) blocks for handling connection

4. G3: ....

5. G4: ....

The number of goroutine is increased on demand, but there is also a maximum value, so the degree of parallelism is controllable. When the request is dense, a worker goroutine may process multiple connection serially.
Wokerchan is reused in the Pool, and the pressure on the GC is much reduced.

Compared to native net/http packages, parallelism is not controllable (may be uncertain, runtime will have control?). ), Goroutine can not be reused, reflected in a request for a goroutine, after the end of the destruction, the pressure on the machine is greater.