Golang Learning--Goroutine User Guide

This is a creation in Article, where the information may have evolved or changed.

Concurrency is the most core competitive function of Golang, Golang is not the thread, but the co-process. What is the difference between a process and a thread? The biggest difference is that the turndown thread is lighter. By default, a process can start with a maximum of 254 threads, and this value can be changed to No limit, but the host resource consumption is very serious. And the use of the association is different, a process can easily start tens of thousands of processes without pressure.

So this article is about how to create a use association in Golang.

The purpose of the Golang design process is to improve concurrency and, on the other hand, to maximize the capacity of multicore CPUs. The Golang built-in scheduler allows each CPU in a multi-core CPU to perform a single process. Through this design, each CPU is fully mobilized to reduce the CPU idle time, improve the CPU throughput, the invisible also increased I/O efficiency.

Referring to the Golang of the process, we have to mention a noun: pipeline (pipeline). Pipelines here are not the same as pipes in a Linux system, and the pipelines here refer to data streams with multi-channel channels that are connected by using the channel to connect multiple processing steps. In general, pipelines read data through an inflow port, send data from a stream exit, and then call certain functions to process the data after reading the data.

Each level in the pipeline can have multiple inflow and flow exits, but the first and last stages of the pipeline generally have only one inflow or flow exit. The head that owns the flow exit is generally called the data source or producer, and the last level of the inflow port is generally called the end point or consumer.

These technical explanations, which look boring, are being explained in more detail through a few simple examples. First look at the following example. In this example, there are three steps to processing the data, and the first gen function is responsible for putting the incoming data into the channel, and when the number is rumored, close the channel. The code is as follows:

func gen(nums ...int) <-chanint {    make(chanint)    gofunc() {        forrange nums {            out <- n        }        close(out)    }()    return out}

In the second step, the SQ function reads the data from the channel and multiplies each value, then sends the data after the operation to the next channel. The code is as follows:

func sq(in <-chanint) <-chanint {    make(chanint)    gofunc() {        forrange in {            out <- n * n        }        close(out)    }()    return out}

The last step is the main function. The main function accepts the data sent in phase two and then outputs the data to know that the channel is closed. The code is as follows:

func main() {    // Set up the pipeline.    c := gen(23)    out := sq(c)    // Consume the output.    fmt.Println(<-out// 4    fmt.Println(<-out// 9}

Because the parameter type and return type of the SQ function are the same, the SQ function can be combined and the modified code is as follows:

func main() {    //and consume the output.    forsq(sq(gen(2, 3))) {        1681    }}

Here, the above three steps complete a very basic Golang concurrency model. But there are many flaws, and we continue to optimize it. The first step is to process a single channel each step, instead of dealing with multiple channel.

In the Golang concurrency model, there are two concepts: fan-in (fan-in) and fan-out (fan-out). Fan-in refers to a program that can read data from multiple channel simultaneously and process it until a definite stop signal is received or all the channel is closed.
Fanout refers to multiple programs that can read and process data from a channel at the same time, until the channel is closed. The larger the fan-out value, the higher the CPU utilization, and the higher the IO utilization rate.

The following optimizations are for fan-in and fan-out.

We will call the SQ function once to call the two sq function and introduce a merge function to fan into the processing result data.

func main() {    in := gen(23)    read from in.    sq(in)    sq(in)    and c2.    for n := range merge(c1, c2) {        49or94    }}

The merge function merges data from multiple channel into a channel by initiating a process. In the Golang language, sending data to an already closed channel causes a run-time exception, so it is necessary to ensure that the channel is not closed before sending the data. Here, we use sync. Waitgroup do synchronization, only the data sent out, the channel will be closed.

FuncMerge(CS ... <-chanint) <-chanint{var wg sync. Waitgroup out: = Make (chanint)    //StartAnOutputGoroutine for  each inputChannelinchCs.OutputCopiesValues  fromC toOut until C isClosed ThenCalls WG. Done.Output: = Func (c <-chanint) { forN: = range C {out <-n} WG. Done ()} WG.ADD(Len (CS)) for_, C: = Range cs {Go Output(c)}//StartA goroutine to CloseOut once AllTheOutputGoroutines isDone. This mustStart  AfterThe WG.ADD Pager.GoFunc () {WG. Wait ()Close(out)} () return out}

Now we have a model like this:

• The channel is closed only after all data has been sent.
• Other processes will continue to accept data until all channel is closed.

With this model, we can iterate over and process the data. But our footsteps will not stop there, let's continue to optimize.

At present, all the processes are independent operation, responsible for the transmission of the process can not stop sending data, accept the data of the process will continue to accept data. What if the data is no longer needed by the process that accepts the data, then how can the upstream process be notified?

In the example above, if an exception occurs at one stage and the other is unable to know the event, some resource leaks occur.

    // Consume the first value from output.    out := merge(c1, c2)    fmt.Println(<-out4or9    return    // Since we didn'tout,    ofisto send it.}

So the direction of the next optimization is the co-operation between the processes. Take the channel first, because the channel can be buffered. So we declare a channel with a buffer:

make(chanint, 2// buffer size 2c <- 1  // succeeds immediatelyc <- 2  // succeeds immediatelyc <- 3  // blocks until another goroutine does <-c and receives 1

The channel buffer is 2, so only two values can be placed at a time, and only those two values are processed before the new value can be placed inside.

In this way, we can modify a gen function.

func gen(nums ...int) <-chanint {    make(chanintlen(nums))    forrange nums {        out <- n    }    close(out)    return out}

Back in the merge function, we can also consider using a buffer channel in the merge function:

...<-chan int) <-chan int {    var wg sync.WaitGroup    1for the unread inputs    ......

It is not a good idea to declare a buffer=1 channel directly. Because this value is now known, but if it changes in the future, then you have to modify the code, so it is best to write generic code. But let's use it this way.

These seem and work together, it's okay. The following is the related code, the addition of the main function is ready to exit, that is, no longer accept the data. The main function needs to notify the upstream of the process to stop sending the data, how does the main function do this?

The main function uses a different channel to do this, and when it needs to exit, main sends a message through the new channel of Done, as follows:

func main () {in : = Gen (2 , 3 ) //distribute the sq work across the Goroutines t Hat both read from in.  C1: = sq (in ) C2: = sq (in ) //consume the first value from output.  done : = Make (chan struct  {}, 2 ) Out: = Merge (done , C1, C2) fmt. Println (<-out) //4 or 9  //tell the remaining senders we ' re leaving.  done  <-struct  {} {} done  <-struct  {} {}} 

Main has sent an empty struct to done, but this has nothing to do with what we care about is whether there is value in done, not what value. If the other process needs to accept the signal, then it needs to use Select to process done.

...<-chan int) <-chan int {    var wg sync.WaitGroup    out := make(chan int)    forin cs.  output    // copies values from c to out until c is closed or it receives a value    // from done, then output calls wg.Done.    output := func(c <-chan int) {        for n := range c {            select {            case out <- n:            case <-done:            }        }        wg.Done()    }    ......

Although this method can achieve the purpose of the notification, but there is a problem: the main function needs to know clearly how many of the co-processes need to be notified, so done <-struct{}{} needs to be called continuously until all the co-processes are notified in place. If some of the process has not been informed, hehe, wait to see the abnormal bar.

To solve this problem, we notify all the processes by shutting down the done way. Accepting data from an already closed channel will cause the current process to exit immediately. So when done is turned off in the main function, all the co-workers waiting to accept the close signal from done will automatically exit.

funcMain () {//Set up-a done channel that' s shared byThe whole pipeline,// andClose thatChannel when this pipeline exits, asA signal// forAll the goroutines we started to exit. Done: = Make (chanstruct{}) Defer close (done)inch: = Gen (Done, 2, 3)//Distribute the sq work across, goroutines thatBoth read from inch. C1: = sq (done,inch) C2: = sq (done,inch)//consume the first value fromOutput. Out: = Merge (Done, C1, C2) fmt. PRINTLN (<-out)//4or9//Done would beClosed byThe deferred call.}

In this way, the merge function can clearly know that its downstream has no longer need to process data, merge can safely exit. The sq can also be learned by knowing that done has been closed and no longer sending data outward. The WG will be called when these functions are exited again. Done to tell main that they are all legally exited.

...<-chan int) <-chan int {    var wg sync.WaitGroup    out := make(chan int)    forin cs.  output    // copies values from c to out until c or done is closed, then calls    // wg.Done.    output := func(c <-chan int) {        defer wg.Done()        for n := range c {            select {            case out <- n:            case <-done:                return            }        }    }    ......

funcSq (Done <-Chan struct{}, in <-Chan int) <-Chan int{out: = Make(Chan int)Go func() {defer Close(out) forN: =Rangein {Select{ CaseOut <-n * N: Case<-done:return}        }    }()returnOut

In this way, it is true that the co-operation between the process is complete.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced. Please pay attention to the original http://blog.csdn.net/vikings_1001