"Go concurrent Programming" chapter II-Goroutines and channels

Goroutines

Goroutine is the most basic execution unit in go. In fact, every go program has at least one goroutine: the main goroutine. When the program starts, it is created automatically.

In fact, Goroutine employs a fork-join model.

sayHello := func() {    fmt.Println("hello")}go sayHello()

So how do we join Goroutine? You need to introduce a wait operation:

var wg sync.WaitGroup()sayHello := func() {    defer wg.Done()    fmt.Println("hello")}wg.Add(1)go sayHello()wa.Wait()

Channel

Read and Write Channel

Goroutine is the basic dispatch unit of the go language, while channels is the communication mechanism between them. The <-operator is used to specify the direction, send, or receive of a pipe. If no direction is specified, it is a two-way pipeline.

// 创建一个双向channelch := make(chan interface{})

interface{} indicates that Chan can be of any type

The channel has two main operations to send and receive. The <-operator is used for both sending and receiving two operations. In the Send statement, the channel is placed to the left of the <-operator. In the receive statement, the channel is placed to the right of the <-operator. A receive operation that does not use the receive result is also legal.

// 发送操作ch <- x // 接收操作x = <-ch // 忽略接收到的值，合法<-ch     

We cannot mistake the direction of the channel:

writeStream := make(chan<- interface{})readStream := make(<-chan interface{})<-writeStreamreadStream <- struct{}{}

Close Channel

The channel supports the close operation to close the channel, and any subsequent send operation to the channel will result in an panic exception . Receiving operations on a channel that has already been close can still receive data that has been successfully sent before, and if there is no data in the channel, it will produce a 0 value data.

Read from the channel that has been closed:

intStream := make(chan int) close(intStream)integer, ok := <- intStreamfmt.Pritf("(%v): %v", ok, integer)// (false): 0

In the above example, by returning the value OK to determine if the channel is closed, we can also handle the closed channel in a more elegant way with range:

intStream := make(chan int) go func() {    defer close(intStream)     for i:=1; i<=5; i++{         intStream <- i     }}()for integer := range intStream {     fmt.Printf("%v ", integer)}// 1 2 3 4 5

Channel with buffer (buffered)

Created a buffered channel that can hold three string elements:

ch = make(chan string, 3)

We can continuously send three values to a newly created channel without blocking:

ch <- "A"ch <- "B"ch <- "C"

At this point, the channel's internal buffer queue will be full, if there is a fourth send operation will be blocked.

If we receive a value:

fmt.Println(<-ch) // "A"

Then the buffer queue for the channel will not be full or empty, so there will be no blocking of the send or receive operations performed on the channel. In this way, the buffer queue buffers of the channel are decoupled from the received and sent Goroutine.

Buffered channels can be used as semaphores, such as limiting throughput. In this case, the incoming request is passed to handle, which receives the value from the channel and sends the value back to the channel after processing the request, so that the semaphore is ready for the next request. The capacity of the channel buffer determines the maximum number of simultaneous calls to process.

var sem = make(chan int, MaxOutstanding)func handle(r *Request) {    sem <- 1 // 等待活动队列清空。    process(r)  // 可能需要很长时间。    <-sem    // 完成；使下一个请求可以运行。}func Serve(queue chan *Request) {    for {        req := <-queue        go handle(req)  // 无需等待 handle 结束。    }}

However, it has a design problem: Although only maxoutstanding Goroutine can run concurrently, Serve creates a new goroutine for each incoming request. The result is that the program consumes resources indefinitely if the request comes soon. To compensate for this deficiency, we can limit the creation of Go by modifying the Serve, which is an obvious solution, but beware of bugs that occur after we fix it.

func Serve(queue chan *Request) {    for req := range queue {        sem <- 1        go func() {            process(req) // 这儿有 Bug，解释见下。            <-sem        }()    }}

The Bug appears in Go's for loop, and the loop variable is reused for each iteration, so the REQ variable is shared across all goroutine, which is not what we want. We need to make sure that req is unique to every goroutine. One way to do this is to pass the value of req as an argument into the closure of the Goroutine:

func Serve(queue chan *Request) {    for req := range queue {        sem <- 1        go func(req *Request) {            process(req)            <-sem        }(req)    }}

Another solution is to create a new variable with the same name, as shown in the example:

func Serve(queue chan *Request) {    for req := range queue {        req := req // 为该 Go 程创建 req 的新实例。        sem <- 1        go func() {            process(req)            <-sem        }()    }}

Let's look at a bible example of Go language. It makes requests to three mirror sites concurrently, and three mirror sites are geographically dispersed. They send the received response to the buffered channel, respectively, and the receiver receives only the first received response, which is the fastest response. Therefore, the Mirroredquery function may return results before two other slow-response mirror sites respond.

func mirroredQuery() string {    responses := make(chan string, 3)    go func() { responses <- request("asia.gopl.io") }()    go func() { responses <- request("europe.gopl.io") }()    go func() { responses <- request("americas.gopl.io") }()    // 仅仅返回最快的那个response    return <-responses }func request(hostname string) (response string) { /* ... */ }

If we use a non-buffered channel, then two slow goroutines will be stuck forever because no one is receiving it. This situation, known as the goroutines leak , will be a bug. Unlike garbage variables, leaked goroutines are not automatically recycled, so it is important to ensure that every goroutine that is no longer needed can exit normally.

Channels of Channels

The most important feature of Go is that the channel is first-class value, which can be assigned and passed around like other values. This feature is often used to implement a secure, parallel, multi-channel decomposition.

We can use this feature to implement a simple RPC.
The following is the approximate definition of the Request type.

type Request struct {    args        []int    f           func([]int) int    resultChan  chan int}

The client provides a function and a reference, and also a channel to receive the reply in the Request object.

func sum(a []int) (s int) {    for _, v := range a {        s += v    }    return}request := &Request{[]int{3, 4, 5}, sum, make(chan int)}// 发送请求clientRequests <- request// 等待回应fmt.Printf("answer: %d\n", <-request.resultChan)

handler functions on the server side:

func handle(queue chan *Request) {    for req := range queue {        req.resultChan <- req.f(req.args)    }}

Channels Pipeline

Channels can also be used to connect multiple goroutine together, and the output of one channel as input to the next channel. This series of channels is called the pipe (pipeline). The following program concatenates three goroutine with two channels:

The first goroutine is a counter for generating 0, 1, 2 、...... The integer sequence of the form, then sends the integer sequence to the second goroutine through the channel; the second goroutine is a square-squared program that squares each integer received. The result of the square is then sent to the third Goroutine by the second channel; the third goroutine is a print program that prints each integer that is received.

func counter(out chan<- int) {    for x := 0; x < 100; x++ {        out <- x    }    close(out)}func squarer(out chan<- int, in <-chan int) {    for v := range in {        out <- v * v    }    close(out)}func printer(in <-chan int) {    for v := range in {        fmt.Println(v)    }}func main() {    naturals := make(chan int)    squares := make(chan int)    go counter(naturals)    go squarer(squares, naturals)    printer(squares)}

Select multiplexing

Select is used to choose a further process from a set of possible communications. If any of the communication can be further processed, then randomly select one to execute the corresponding statement. Otherwise, if there is no default branch, the SELECT statement blocks until one of the communications is complete.

select {case <-ch1:    // ...case x := <-ch2:    // ...use x...case ch3 <- y:    // ...default:    // ...}

How to use a SELECT statement to set a time limit for an operation. The code outputs the value of the variable news or the timeout message, depending on which of the two receive statements is executed first:

select {case news := <-NewsAgency:    fmt.Println(news)case <-time.After(time.Minute):    fmt.Println("Time out: no news in one minute.")}

The following SELECT statement receives a value from the Abort channel when it has a value, and does nothing when there is no value. This is a non-blocking receive operation; Doing this repeatedly is called "polling channel."

select {case <-abort:    fmt.Printf("Launch aborted!\n")    returndefault:    // do nothing}

references.

1. Concurrency in Go
2. Gopl
3. Effective Go