Golang Network: Core API Implementation Analysis II)

This is a creation in Article, where the information may have evolved or changed.

Description

In the previous chapters we basically finished the key API flow for Golang network programming, but left behind a key element: When the system call returns Eagain, it calls Waitread/waitwrite to block the current process, and now we're going to talk.

Waitread/waitwrite

func (pd *pollDesc) Wait(mode int) error {    res := runtime_pollWait(pd.runtimeCtx, mode)    return convertErr(res)}func (pd *pollDesc) WaitRead() error {    return pd.Wait('r')}func (pd *pollDesc) WaitWrite() error {    return pd.Wait('w')}

Eventually Runtime_pollwait went down to the following:

TEXT net·runtime_pollWait(SB),NOSPLIT,$0-0    JMP runtime·netpollWait(SB)

We carefully consider that the main function of the netpollwait is to wait for the socket to be concerned about whether there is an event (in fact, we know just wait for a mark bit to change), if there is no event, then the current process is suspended until a notification event occurs, Let's take a look at how this is achieved:

Func netpollwait (PD *polldesc, mode int) int {//first check if the socket has an error (such as shutdown, timeout, etc.) Err: = Netpollcheckerr (PD, Int32 (Mo     DE)) If err! = 0 {return err}//As for today only Solaris uses level-triggered IO.     if GOOS = = "Solaris" {OnM (func () {netpollarm (PD, Mode)})}//Loop wait Netpollblock return value True If the return value is false and the socket does not have any errors//then the thread may wake unexpectedly and need to be re-suspended//Another possibility: the socket was woken up due to a timeout//This time Netpollcheckerr is used to detect timeouts            Error for!netpollblock (PD, Int32 (mode), false) {err = Netpollcheckerr (PD, Int32 (Mode)) If Err! = 0 { Return err}} return 0}func netpollblock (PD *polldesc, mode int32, waitio bool) bool {GPP: = &A     Mp;pd.rg if mode = = ' W ' {gpp = &PD.WG}//Set the GPP semaphore to WAIT//first set polling status to Pdwait Why use for?     Because CASUINTPTR uses spin lock//Why use spin lock to add for loop?      for {old: = *GPP if old = = Pdready {*gpp = 0 return True   } if old! = 0 {gothrow ("netpollblock:double Wait")}//Set the socket polling related status to Pdwait  If Casuintptr (GPP, 0, pdwait) {break}}//If the coprocessor is not faulted, the unlock function is netpollblockcommit if waitio || Netpollcheckerr (PD, mode) = = 0 {f: = Netpollblockcommit gopark (* * (**unsafe). Pointer) (unsafe. Pointer (&f)), unsafe. Pointer (GPP), "IO Wait")}//may be suspended by the process is woken//or for some reason the coprocessor is not at all suspended//gets its current state record in old: = Xchguintptr (GPP , 0) If old > pdwait {gothrow (' netpollblock:corrupted state ')} return old = = Pdready}

From the above analysis, we can see that if you cannot read and write, Golang will suspend the current process, and when the process is woken up, the token bit should be placed. Let's take a look at when these pending threads are awakened.

Event notification

The Golang runtime has a socket event checkpoint while the system is running, and the checkpoint is currently located mainly in the following places:

runtime Starttheworldwithsema (void): After the GC has been completed;
Findrunnable (): Does this temporarily not know when will trigger?
The monitoring process in the Sysmon:golang will periodically check the ready socket

TODO: Why check for Socket readiness events in these places?

Let's look at how to check the socket readiness event and how to wake up the suspended thread when the socket is ready? Main Call Function Runtime-netpoll ()

We only focus on the implementation of Epoll, for Epoll, the above method is netpoll_epoll.go in the implementation of the Netpoll

Func netpoll (block bool) (GP *g) {if epfd = =-1 {return} WAITMS: = Int32 ( -1) if!block {//     If the caller does not want block//set WAITSM to 0 Waitms = 0} var events [128]epolleventretry://Call epoll_wait Get ready Event    N: = epollwait (EPFD, &events[0], Int32 (Len (Events)), WAITMS) if n < 0 {...} Goto Retry} for I: = Int32 (0); I < n; i++ {ev: = &events[i] if ev.events = = 0 {Continue} var mode int32 I F ev.events& (_epollin|_epollrdhup|_epollhup|_epollerr)! = 0 {mode + = ' r '} if Ev.events&amp ;(_epollout|_epollhup|_epollerr)! = 0 {mode + = ' W '}//For each event, the Netpollready//PD main record is called The wait coprocessor associated with the socket if mode! = 0 {PD: = * (**POLLDESC) (unsafe. Pointer (&ev.data)) Netpollready ((**g) (Noescape (unsafe.     Pointer (&AMP;GP))), PD, Mode)}}//If the caller is waiting synchronously and this time does not get to the ready socket//Continue retryingIf block && GP = = nil {goto retry} return GP} 

This function primarily calls epoll_wait (of course, the Golang encapsulates the system call) to get the ready socket FD, which is further processed for each ready FD, called Netpollready (). The final return value of this function is a ready-to-Go (g) list.

Netpollready is mainly to mark the socket FD as Ioready, and to wake up the process G waiting on the FD to add it to the incoming G-linked list.

Make PD ready, newly runnable goroutines (if any) is returned in RG/WG func netpollready (gpp **g, PD *polldesc, mode I Nt32) {var RG, WG *g If mode = = ' R ' | | mode = = ' R ' + ' W ' {RG = Netpollunblock (PD, ' R ', true)} if mode = = ' W ' | | mode = = ' R ' + ' W ' {WG = Netpollunblock (PD, ' W ', True)}///will be ready to be added to the linked list if RG! = Nil {Rg.schedlink = *GPP *GPP = RG} If WG = nil {Wg.schedlink = *gpp *GPP = WG}}//set POLLDESC status to Pdready and return  Back-ready coprocessor func Netpollunblock (PD *polldesc, mode int32, Ioready bool) *g {GPP: = &pd.rg if mode = = ' W ' {GPP  = &pd.wg} for {old: = *GPP if old = = Pdready {return nil} if old = = 0 &&!ioready {return nil} var new uintptr if Ioready {new = Pdre             Ady} if Casuintptr (GPP, old, new) {if old = = Pdready | | old = pdwait {old = 0 }           Return (*G) (unsafe. Pointer (Old)}}}

Question: Will an FD be simultaneously IO by multiple concurrent processes? For example, a co-reading, another process to write? or multiple simultaneous reads? What is the process ready to return at this time?

A socket FD can support concurrent read and write because it is full-duplex for the TCP protocol. Read and write operations are different buffers, but do not support concurrent read and concurrent write, because this will be chaotic. So the above Netfd.rwlock () is the function of this.