The implementation of Go channel

This is a creation in Article, where the information may have evolved or changed.

Channel, as an important way of communication and synchronization between Goroutine, is an important member of the Go Runtime layer to implement CSP concurrency model. When you do not understand the underlying implementation, you often have doubts about the performance of the Channe-related syntax in use, especially the behavior of Select Case. So take a look at the channel implementation before you understand the application of the channel.

Channel Memory layout

The channel is the built-in type of go, which can be stored in a variable, and can be used as a function parameter or return value, which is Hchanin the data structure of the runtime layer. Hchan maintained a list of two linked lists, and Recvq was blocked by reading this Chan G,SENDQ the G that was blocked by writing this Chan. The data structure for each element in the WAITQ queue is Sudog, where elem is used to hold the information.

type hchan struct {    qcount   uint           // total data in the queue    dataqsiz uint           // size of the circular queue    buf      unsafe.Pointer // points to an array of dataqsiz elements    elemsize uint16    closed   uint32    elemtype *_type // element type    sendx    uint   // send index    recvx    uint   // receive index    recvq    waitq  // list of recv waiters    sendq    waitq  // list of send waiters    lock     mutex}type sudog struct {    g           *g    selectdone  *uint32    next        *sudog    prev        *sudog    elem        unsafe.Pointer // data element    releasetime int64    nrelease    int32  // -1 for acquire    waitlink    *sudog // g.waiting list}

The Hchan is just the head of the channel, and a contiguous array of memory behind the head acts as a buffer for the channel, the ring queue used to hold the channel data . Qcount DataSize describes the current usage and capacity of buffers respectively. If the channel is unbuffered, the size is 0 and there is no such ring queue.

Creating Chan requires knowing the data type and buffer size. The corresponding structure diagram newarray will generate this ring queue. The reason to separate the pointer type buffers is primarily to differentiate the GC operation, which needs to be set to Flagnoscan. And the pointer size is fixed, you can allocate memory with Hchan head, no need to first new(hchan) newarry .

Chan, which declares but does not initialize, is nil Chan. Read/write Nil Chan will block and close Nil Chan will panic.

func makechan(t *chantype, size int64) *hchan {    elem := t.elem    var c *hchan    if elem.kind&kindNoPointers != 0 || size == 0 {    c = (*hchan)(mallocgc(hchanSize+uintptr(size)*uintptr(elem.size),     nil, flagNoScan))        if size > 0 && elem.size != 0 {            c.buf = add(unsafe.Pointer(c), hchanSize)        } else {            c.buf = unsafe.Pointer(c)        }    } else {        c = new(hchan)        c.buf = newarray(elem, uintptr(size))    }    c.elemsize = uint16(elem.size)    c.elemtype = elem    c.dataqsiz = uint(size)    return c}

Channel operations

Read and write from the implementation of Chan all to lock, which is the same as read and write shared memory has the lock overhead.

The direction in which the data is passed from Chansend begins with the data field of the Goroutine in the RECVQ, which, if blocking occurs, may be written to the Goroutine data domain in SENDQ to wait for transit.

Sudog objects can be reused after returning from Gopark.

Synchronous Read and Write

Write channel c<-x calls runtime.chansend . Read Channel <-c calls runtime.chanrecv . The process of summarizing synchronous reading and writing is:

• When you write Chan, you have to check the RECVQ for the goroutine of Chan, if there is a sudog from RECVQ. syncsendthe Data EP that will be written to Chan is copied to the Elem domain of the sudog that was just out of the team. By goready Waking the receiver G, the status is set to _Grunnable , and then put in the P local to run queue. After this the read to the data of G can again be dispatched by P.
• When writing Chan, if there is no g waiting to be read, current g is blocked by waiting for writing. At this point acquireSudog , the data to be written on the package is created or fetched into the SENDQ queue. At the same time the current G gopark sleep wait is awakened.
• When you read Chan, you first wake up the goroutine waiting to be written in the SENDQ and get the data from it, and if no one writes, hang yourself in the recvq waiting to wake up.

 func chansend (t *chantype, C *hchan, ep unsafe.)        Pointer, block bool, CALLERPC uintptr) bool {... lock (&c.lock) If C.dataqsiz = = 0 {//synchronous channel SG: = C.recvq.dequeue () if SG! = nil {//found a waiting receiver unlock (&c.lock) recv G: = Sg.g syncsend (c, SG, EP) Goready (RECVG, 3) return true}//No Receive        R Available:block on the This channel. MYSG: = Acquiresudog () Mysg.elem = EP C.sendq.enqueue (MYSG) goparkunlock (&c.lock, "Chan send",        Traceevgoblocksend, 3)//Someone woke us up. Releasesudog (MYSG) return True}}  

func chanrecv(t *chantype, c *hchan, ep unsafe.Pointer, block bool) (selected, received bool) {    if c.dataqsiz == 0 { // synchronous channel        sg := c.sendq.dequeue()        if sg != nil {            unlock(&c.lock)            typedmemmove(c.elemtype, ep, sg.elem)            gp.param = unsafe.Pointer(sg)            goready(gp, 3)            return true, true        }        // no sender available: block on this channel.        mysg := acquireSudog()        mysg.elem = ep        c.recvq.enqueue(mysg)        goparkunlock(&c.lock, "chan receive", traceEvGoBlockRecv, 3)        // someone woke us up        releaseSudog(mysg)        return recvclosed(c, ep)    }}

asynchronous read and write

The difference between asynchronous and synchronous is that the read-write priority is to check that the buffer has no data read or no space to write. And when you actually read and write to Chan, a buffer change occurs, and the goroutine that might have previously been blocked will have the opportunity to write and read, so try to wake them up. Summary process:

When
• writes Chan, the buffer is full, and the current G and data are encapsulated into the SENDQ queue waiting to be written, while the Gopark current goroutine is suspended. If the buffer is not full, the data is written directly to the buffer and the index and qcount of the latest buffer data are updated. At the same time, try to wake Goready from RECVQ with a wait-read goroutine that was previously blocked by a buffer that is not readable. The
• reads Chan first to see if the buffer has no data, if any, to read directly, and to attempt to wake up a wait-write that was blocked by a full buffer before goroutine, giving it the opportunity to write the data. If there is no data to read the queue recvq.

 func chansend (t *chantype, C *hchan, ep unsafe.) Pointer, block bool, CALLERPC uintptr) bool {//asynchronous channel Var t1 int64 for futile: = byte (0); C.qcoun T >= C.dataqsiz; Futile = tracefutilewakeup {mysg: = Acquiresudog () c.sendq.enqueue (MYSG) goparkunlock (&c.lock, " Chan send ", Traceevgoblocksend|futile, 3)//Someone woke US up-try again Releasesudog (MYSG)}//WR ITE our data into the channel buffer Typedmemmove (C.elemtype, Chanbuf (c, C.sendx), EP) c.sendx++ if c.sendx = = c. dataqsiz {c.sendx = 0} c.qcount++//Wake up a waiting receiver sg: = C.recvq.dequeue () if SG! = N Il {goready (SG.G, 3)} return true}  

func chanrecv(t *chantype, c *hchan, ep unsafe.Pointer, block bool) (selected, received bool) {    // asynchronous channel    for futile := byte(0); c.qcount <= 0; futile = traceFutileWakeup {        mysg := acquireSudog()        c.recvq.enqueue(mysg)        goparkunlock(&c.lock, "chan receive", traceEvGoBlockRecv|futile, 3)        // someone woke us up - try again        releaseSudog(mysg)    }    typedmemmove(c.elemtype, ep, chanbuf(c, c.recvx))    memclr(chanbuf(c, c.recvx), uintptr(c.elemsize))    c.recvx++    if c.recvx == c.dataqsiz {        c.recvx = 0    }    c.qcount--    // ping a sender now that there is space    sg := c.sendq.dequeue()    if sg != nil {        goready(sg.g, 3)    }    return true, true}

Shut down

Wake up the goroutine waiting to be read in RECVQ by Goready, then wake up Goroutine waiting to be written in all SENDQ. so Close Chan is the equivalent of releasing all gouroutine that are blocked by it.

func closechan(c *hchan) {    c.closed = 1    // release all readers    for {        sg := c.recvq.dequeue()        if sg == nil {            break        }...        goready(gp, 3)    }    // release all writers    for {        sg := c.sendq.dequeue()        if sg == nil {            break        }...        goready(gp, 3)    }}

Writing closed Chan or closing closed Chan will lead to panic. Read closed Chan will never block, return a 0 value of a channel data type, and return the parameter EP to the function.

Therefore, it is often necessary to read in close Chan to determine if Chan is closed.

if v, open := <- c; !open {   // chan is closed}

Happens before

It's interesting to talk about the happens-before problem in go memory model. Some of these sync rules that are related to Chan can explain some of the questions that have always been recorded as follows:

• Write operation with buffered Chan Happens-before corresponding Chan's read operation
• Close Chan Happens-before read the last return value from the Chan 0
• Non-buffered Chan's read operation Happens-before corresponding Chan's write operation

var c = make(chan int, 10)var a stringfunc f() {  a = "hello, world"  //(1)  c <- 0  // (2)}func main() {  go f()  <- c  //(3)  print(a)  //(4)}

(1) Happens-before (2) (3) Happens-before (4), then according to the rules (2) happens (3). So (1) Happens-before (4), this code is no problem, will definitely output Hello world.

var c = make(chan int)var a stringfunc f() {  a = "hello, world"  //(1)  <-c  // (2)}func main() {  go f()  c <- 0  //(3)  print(a)  //(4)}

Also according to the rule three (2) Happens-before (3) can finally be guaranteed (1) happens-before (4). If C is changed to a buffered Chan, the result will no longer have any synchronization guarantees (2) Happens-before (3).