Further understanding of concurrency in Golang

This is a creation in Article, where the information may have evolved or changed.

If you are a programmer, then the term concurrency must be very familiar to you. There are too many scenarios that require concurrency, such as a chat program, and if you want the chat program to be able to receive messages and send messages at the same time, it will certainly use concurrency, no matter what the concurrency is.

The meaning of concurrency is: Let a program do multiple things at the same time!

It's important to understand that, yes, concurrency is just to get the program to do something else at the same time, and the purpose of concurrency is not to make the program run faster (if it's a multicore processor, and the tasks can be split into separate parts, then concurrency can actually make things faster). Remember when I learned C + + when I started to touch concurrency, I thought every one of the threads would speed up a few times ....

Golang provides support for concurrency from the language level and adds language-level keywords directly to the way it initiates concurrency. I don't speak a lot of languages, and I don't have a lot of project experience, and maybe the comparison from my mouth is not very objective, but at least the concurrency mechanism of Golang is very comfortable when compared to the way the C++11 uses the system API to manipulate threads. You do not have to define thread functions in a fixed format, and you do not have to worry about having to pass a parameter to a thread function only when you start the thread. Compared to Java, the advantage of GO is that the concurrent parts do not have to be implemented into a class, and lighter (in fact, I do not know why the lighter ^_^).

Since I recently wanted to write a small open source project, and the key part of it used a lot of concurrency, I started to revisit the concurrency-related knowledge of learning go. From my study go to now has been nearly 1 years, I think now re-see go concurrency when the harvest a lot, because after all, wrote many go of small program, encountered many unexplained phenomenon and confusion, through this restudying opportunity, learn new experience hurriedly record down, Share to all of you, especially friends who like go.

Concurrent start-up

This article about the concurrent start I was all over, if you want a function to run concurrently, just a keyword "go":

Func afuntion (Para1, Para2, Para3, ...) {//Do some process//...} Func main () {Go afuntion (para1, Para2, PARA3, ...)//Just add a go prefix and afunction () will run concurrently}

the concurrent start of Go is very simple, there is little extra preparation, there is no difference between the function and the general function, the parameters are arbitrary, the start time only need to add a go key.

Of course, concurrent booting is nothing to say, the most essential part of concurrency is the scheduling of these processes (which are similar to threads, but are lighter threads).

I can't tell you all aspects of scheduling with a veteran expert, but I can share some of the scenarios I've encountered with the scheduling methods I've used (so it's absolutely possible).

Go provides the Sync packet and channel mechanism to resolve synchronization and communication between the threads. The use of the channel is very flexible and varied in the way it is used, and the effective go of the official website gives a way out of the channel's concurrency. Let's start by introducing the scheduling support provided by the sync package.

Sync. Waitgroup

The waitgroup in the sync package implements a similar task queue structure where you can add tasks to the queue, remove the task from the queue when the task is complete, and if the tasks in the queue are not all complete, the queue will trigger blocking to prevent the program from continuing to run. For specific usage, refer to the following code:

Code sticky can run through the package Mainimport ("FMT" "Sync") var waitgroup sync. Waitgroupfunc afunction (shownum int) {FMT. Println (Shownum) waitgroup. Done ()//task completed, number of tasks in the task queue-1, in fact. Done is. ADD ( -1)}func main () {for i: = 0; i <; i++ {Waitgroup. ADD (1)//Each goroutine is created, the number of tasks in the task queue +1go afunction (i)}waitgroup. Wait ()//. Wait () This will block until all the tasks in the queue are ended and unblocked}

we can use sync. Waitgroup to meet this situation:

▲ There is a place where you need to create multiple goroutine, and be sure to wait until they are all done before you proceed to the next step.

Yes, the biggest advantage of Waitgroup is that. Wait () can block until the task in the queue is complete before unblocking.

Channel
Channel is a kind of golang built-in type, the literal translation of English is "channel", in fact, it is really a pipe, and is a first-out data structure.

There are only 4 ways we can operate on the channel:

(1) Create Chennel (via make () function)

(2) put data (via channel <-)

(3) Remove data (via <-channel operation)

(4) Close channel (via Close () function)

But the channel has some very force-giving properties that you need to keep in mind, and be sure to remember and understand them:

(1) channel is a kind of blocking pipeline, which is automatically blocked. This means that if the pipe is full, an operation that puts data into the channel will block until a certain routine is taken out of the channel and the data is executed. Conversely, if the pipe is empty, an operation that pulls the data out of the channel will block until a routine puts data into the channel, and the operation to fetch the data will be executed. This is one of the most important nature of the channel, none of it .

Package MainFunc Main () {ch: = make (chan int, 3) ch <-1ch <-1ch <-1ch <-1//This line of operation will be blocked, because the first three rows of the data into the operation has been the CHA Nnel filled up}

Package MainFunc Main () {ch: = make (chan int, 3) <-ch//This line is blocked because the channel is just created, is empty, nothing can be removed}

(2) channel is divided into buffered channel and non-buffered channel. the two channel methods are created as follows:

CH: = make (chan int)//unbuffered channel, equal to make (chan int, 0) ch: = Make (chan int, 5)//channel with buffer size 5

when manipulating a channel, it is important to note that it is buffered because some operations trigger a deadlock in the channel's blocking. Here's a look at the scenarios that need attention.

Let's start with an example, which is a two-part code with a different main function:

Package Mainimport "FMT" func afuntion (ch Chan int) {FMT. Println ("Finish") <-ch}func main () {ch: = make (chan int)//unbuffered Channelgo afuntion (CH) ch <-1//Output Result://Finish}

Package Mainimport "FMT" func afuntion (ch Chan int) {FMT. Println ("Finish") <-ch}func main () {ch: = make (chan int)//unbuffered channel//just swap the code order of the two lines for ch <-1go afuntion (CH)// Output Result://deadlock, no result}

The previous code eventually outputs "finish" and ends normally, but a deadlock occurs in the latter section of the code. Why is this happening, let's run the logic of the above two pieces of code.

First paragraph of code:

1. Created a non-buffered channel

2. A goroutine is activated, the routine is taken out of the channel, but because the channel is empty at this time, the fetch operation is blocked, but the primary routine is not blocked, and it continues to run.

3. The main goroutine this time continue to execute the next line, the channel into a data

4. The blocked routine detects the presence of data in the channel, so the contact is blocked, the data is removed from the channel, and the program is completed.

The second piece of code:

1. Created a non-buffered channel

2. The main routine is going to put a data into the channel, but because the channel is not buffered, it is equivalent to the channel is always full, so there will be blocking. But the following goroutine has not been created yet, the main routine here block, the whole program can only be blocked so long, and then ... And then there's no more. Dead lock!

※ As can be seen from here, for unbuffered Channel, the put operation and the removal operation can no longer be in the same routine, and should be to ensure that there is a routine to take out the operation, and then in another routine to perform the put operation.

For the channel with buffering, there is not so much attention, because there is buffer space, so as long as the buffer is not satisfied, put the operation will not block, similarly, as long as the buffer is not empty, the removal operation will not block. Furthermore, the placement and removal of the buffered channel can be used in the same routine.

However, it is not said that with the buffer can be arbitrary use of the channel into and out, we must pay attention to the problem of the rate of putting in and out . Let's give an example to illustrate the problem:

We often use the nature of the channel automatic blocking to control the total number of Goroutine currently running, as follows:

Package Mainimport ("FMT") func afunction (ch Chan int) {FMT. Println ("Finish") <-ch//goroutine takes out a data from the channel when it is finished}func main () {ch: = make (chan int, ten) for I: = 0; i <; i++ {//each time a goroutine is created, a data is placed into the channel, and if there is already 10 data in it, it will//block, thus we will control the total number of simultaneous goroutine in the range of <=10 ch <-1go Afunction (CH)}//Here is just an example of, of course, there should be some more elaborate synchronization operation}

The use of this channel is almost always used, but look at the next piece of code, which is almost the same as the way the channel is used, but it can cause problems:

Package MainFunc afunction (ch chan int) {ch <-1ch <-1ch <-1ch <-1ch <-1<-ch}func Main () {//main routine Operation with the above code ch: = Make (chan int, ten) for I: = 0; I < 100; i++ {ch <-1go afunction (CH)}//This code runs the result of deadlock}

The above operation is basically the same as the previous paragraph, but a deadlock occurs after the operation. Why is it? In fact summed up on a sentence, "Put too fast, get too slow."

supposedly, we should create a sub-goroutine in our main routine and put the data in the channel each time, while the child goroutine is responsible for extracting the data from the channel. But after our code has created the sub-goroutine, each routine will put 5 data into the channel. in this way, 6 data is placed in each channel to perform a fetch operation, so there may be a moment when the channel is full, but all the routine are performing the put operation ( Because the probability that they are currently performing a drop operation is 6 times times the fetch operation, all routine are blocked, resulting in a deadlock.

When using buffered channel, be sure to pay attention to the problem of the rate of putting and removing.

(3) The channel after closing can fetch data, but cannot put data. Also, the channel does not actually close after the close () is executed, and the data in the channel is removed before it is actually closed.

Package MainFunc Main () {ch: = make (chan int, 5) ch <-1ch <-1close (CH) ch <-1//Can not be put into operation on closed channel                //Will trigger PA Nic

Package MainFunc Main () {ch: = make (chan int, 5) ch <-1ch <-1close (CH) <-ch//As long as the channel has data, it is possible to perform a fetch operation        //Normal End}

Package Mainimport "FMT" Func Main () {ch: = make (chan int, 5) ch <-1ch <-1ch <-1ch <-1close (CH)  //if C is executed Lose () immediately closes the channel, the following loop will not have any output for {data, OK: = <-chif!ok {break}fmt. PRINTLN (data)}//output://1//1//1//1////Call Close (), channel is closed only if channel is empty}

use Channel to control the number of Goroutine

The nature of the channel has been introduced here, but it appears that the use of the channel seems to pay more attention to details than waitgroup, so what reason must be to use channel to achieve synchronization? Channel compared to Waitgroup has a great advantage, is that the channel can not only achieve the synchronization of the process, and can control the total number of currently running Goroutine.

Here are a few ways to control the number of Goroutine using channel:

I. If the number of tasks is fixed:

Package MainFunc afunction (ch chan int) {ch <-1}func main () {var (ch        chan int = make (chan int, 20)//can run at the same time Routin The number of E is 20dutycount int      = $) for I: = 0; i < Dutycount; i++ {go afunction (CH)}//knows the total amount of the task, and can use a loop of fixed cycles to detect if all routine are working for I: = 0; i < Dutycount; i++ {<-ch}}

Two. If the number of tasks is not fixed

Package Mainimport ("FMT") func afunction (Routinecontrol Chan int, feedback chan string) {defer func () {<-routinecontro Lfeedback <-"Finish"} ()//Do some process//...} Func main () {var (routinectl chan int    = make (chan int, a) feedback   chan string = Make (Chan string, 10000) msg      s Tringallwork  intfinished int) for I: = 0; i < 1000; i++ {routinectl <-1allwork++go afunction (routinectl, feedback)}for {msg = <-feedbackif msg = = "Finish" {finished++} If finished = = allwork {break}}}

If reproduced please specify the source: http://blog.csdn.net/gophers/article/details/24665419