reprinted from: Http://www.woola.net/detail/2017-04-27-goroutines.html
This article is an introductory article on concurrent programming, writing sample code in the Go language, covering:
run-time threadRun-time concurrent Threads (Goroutines) Basic synchronization techniques (pipelines and locks) basic concurrency Pattern deadlock and data competition parallel computing in Go language
The statement behind the GO keyword will run as a new thread, and what's the difference between this thread and the Java thread we'll explain later
Go allows you to use the GO statement to open a new run-time thread, Goroutine, to execute a function with a different, newly created goroutine. All goroutine in the same program share the same address space.
The Goroutine is very lightweight , except for the stack space allocated to it, which occupies a negligible amount of memory space. and its stack space is very small at the beginning, and then changes as the heap storage space is allocated or released.
On the internal implementation, Goroutine will be multiplexed on multiple operating system threads (if the current thread is blocked, it will jump to other threads to execute). If a goroutine blocks an operating system thread,
For example: Waiting for input, other goroutine on this thread will be migrated to another thread, so that it can continue to run. Developers do not need to care/worry about these details.
The program shown below will output "Hello from main Goroutine". It is also possible to output "Hello from another Goroutine", depending on which of the two goroutine ends first.
The following code fragment, which is not complete, may need to be introduced in the test package.
Func Main () {
go fmt. Println ("Hello from another Goroutine")//thread
FMT. Println ("Hello from Main goroutine")
//This is the end of the program run,//
all active goroutine killed
}The next program, in most cases, will output "Hello from main goroutine" and "Hello from another Goroutine", the order of the output is indeterminate.
But there is another possibility: the second goroutine runs extremely slowly, and before the program ends, it is not possible to output the corresponding message.
Func Main () {
go fmt. Println ("Hello from another Goroutine")
FMT. Println ("Hello from main goroutine") time
. Sleep (time. Second) //wait 1 seconds, wait for another goroutine to end
}The following is a relatively more practical example, where a function is defined that uses concurrency to defer triggering an event.
The function Publish prints the text string to standard output//After a given time expires
and returns the
func Publish (text string, delay time) without blocking. Duration) {
go func () {Time
. Sleep (delay)
FMT. Println ("Breaking NEWS:", Text)
} () //Note the parentheses here. Anonymous function must be called
}You might use the Publish function like this:
Func Main () {
Publish ("A goroutine starts a new thread of execution.", 5*time. Second)
FMT. PRINTLN ("Let's hope the news would published before I leave.")
Waiting to publish news time
. Sleep (Ten * time. Second)
FMT. Println ("Ten seconds later:i ' m leaving now.")
}This program, in most cases, outputs the following three lines, fixed sequentially, and 5 seconds apart from each line of output.
$ go Run Publish1.go Let's hope the news would published before I leave. Breaking news:a goroutine starts A new thread of execution. Ten seconds later:i ' m leaving now.
In general, it is not possible to orchestrate the wait between threads by means of sleep. The next section describes a synchronization mechanism in the go language-the pipeline , and shows how to use a pipe to let one goroutine wait for another goroutine. Pipeline (channel) Chnnel is not a thread, it is a tool used to pass information. The pipeline is a component of the go language, providing a mechanism for synchronizing runs and communication between two goroutine by passing a value of a specified type. The <-operator is used to specify the direction, send, or receive of a pipe. If no direction is specified, it is a two-way pipeline. Chan Sushi//Can be used to send and receive values of Sushi type Chan <-float64//can only be used to send values of float64 type <-chan int//Only available to receive values of type int
Remember to define the channel when it is directed to send, instead pointing out is to accept.
Pipelines are reference types and are allocated based on the Make function.
* IC: = make (chan int) //non-buffered int type pipe * WC: = Make (chan *work, Ten) //buffered work Type pointer pipe
If a value is sent through a pipe, <-is used as a two-dollar operator. To receive a value from a pipeline, use it as a unary operator:
If the pipeline is not buffered, the sender blocks until the receiver receives the value from the pipeline. (Huh ...) If the pipeline is buffered, the sender blocks until the sent value is copied into the buffer; that is, if the message is not being used in the pipeline, then the thread will always block if the buffer is full, it means waiting until a receiver gets a value. The same receiver will block until a value can be received. Close the pipe (close)IC <-3 //Send 3 work to pipeline : = <-WC //Receive a pointer to the work type value from the pipeline
The close function indicates that a value is no longer sent to a pipe. After you call close, and the previously sent values are received, the receive operation returns a value of 0 and is not blocked. A receive operation with a multiple return value returns an extra Boolean value that indicates whether the returned value is passed by the send operation.
CH: = Make (chan string)
go func () {
ch <-"hello!"
Close (CH)
} ()
FMT. Println (<-CH) //output string "hello!"
Fmt. Println (<-CH) //Output 0 value-empty string "", does not block
FMT. Println (<-CH) ///Print output empty string ""
V, OK: = <-ch //The value of the variable v is an empty string "", the value of the variable OK is falseA for statement with a range clause reads the value sent to the pipe sequentially until the pipe is closed:
Func Main () {
//: To run the example, you need to first define the type Sushi, such as the type Sushi string
var ch <-chan Sushi = Producer () for
s: = RA Nge Ch {
FMT. Println ("Consumed", s)
}
}
func Producer () <-chan Sushi {
ch: = Make (chan Sushi)
go func () {
ch <-Sushi ("Sea old Grip moistened") //Ebi nigiri
ch <-Sushi ("Tuna とろ grip moistened")//Toro nigiri
Close (CH)
} ()
return CH
}Anyway, remember that the <-channelname line is waiting for the thread to be close off.
In the next example, we let the publish function return a pipe-to broadcast a message when the value of the text variable is published:
When a given time expires, the Publish function prints the value of the text variable to standard output
//After the text variable value is published, the function closes the pipe wait
func Publish (text string, delay time. Duration) (Wait <-chan struct{}) {
ch: = Make (chan struct{})
go func () {Time
. Sleep (delay)
FMT. Println ("Breaking NEWS:", text)
Close (CH) //broadcast-a closed pipe will send a 0 value
} ()
return CH
}Note: We used a pipe with an empty struct: struct{}. This clearly indicates that the pipeline is used only to signal, not to pass data.
We may use this function in this way:
Func Main () {
wait: = Publish ("Channels let Goroutines communicate.", 5*time. Second)
FMT. Println ("Waiting for the News ...")
<-wait//wait for the end or the news after the detailed input program exits the
FMT. PRINTLN ("The news is out, time to leave.")
}This program outputs the following three lines in the order specified. The last line will be output as soon as the news is out.
dead Lock$ go Run publish2.go waiting for the news ... Breaking news:channels let Goroutines communicate. The news is out and time to leave.
Anyway remember no close (channel) causes another thread to wait
Now let's introduce a bug in the Publish function:
Func Publish (text string, delay time. Duration) (Wait <-chan struct{}) {
ch: = Make (chan struct{})
go func () {Time
. Sleep (delay)
FMT. Println ("Breaking NEWS:", text)
//: Note that the close function call is commented out here
//Close (CH)//No Close
} ()
return CH
}The main program is still running as before: output the first line, and then wait 5 seconds, then the Publish function open Goroutine will output breaking news (breaking), and then exit, leaving the main goroutine alone waiting.
Func Main () {
wait: = Publish ("Channels let Goroutines communicate.", 5*time. Second)
FMT. Println ("Waiting for the News ...")
//: Note The following sentence
<-wait
fmt. PRINTLN ("The news is out, time to leave.")
}After this moment, the program can no longer continue to execute. As we all know, this situation is a deadlock.
Deadlocks are situations in which threads wait for each other and none of them can run forward.
The go language is very well supported for runtime deadlock detection. The Go program usually provides detailed error information when no goroutine is able to proceed forward. The following is the output of our problem program:
Waiting for the news ...
Breaking news:channels let Goroutines communicate.
Fatal Error:all Goroutines is asleep-deadlock!
Goroutine 1 [Chan receive]:
main.main () ...
/goroutinestop.go:11 +0xf6
goroutine 2 [syscall]:
created by runtime.main ...
/go/src/pkg/runtime/proc.c:225
goroutine 4 [Timer goroutine (idle)]:
created by AddTimer ...
/go/src/pkg/runtime/ztime_linux_amd64.c:73In most cases it is easier to find the cause of the deadlock in the Go program, so the rest is how to solve the bug. Data Competition (race)
Deadlocks may sound sad, but the real catastrophic error associated with concurrent programming is data competition, which is quite common and can be very difficult to debug.
When two threads concurrently access the same variable, and at least one of the accesses is write, data contention occurs.
The following function has a data competition problem and its behavior is undefined. For example, the value 1 may be output. The code is followed by a possibility explanation, trying to figure out how it all happened.
Func race () {
wait: = Make (chan struct{})
N: = 0
go func () {
///: Note The following line
n++//One-time access: Read, increment, write
c Lose (Wait)
} ()
//: Note The following line
n++//Another conflicting access
<-wait
fmt. PRINTLN (n)//output: not specified
}N++ actually not directly in memory +1 but put n into the CPU level two cache, processed and then stored in memory
The two goroutine in the code (assuming named G1 and G2) participated in a competition, and we were unable to know in what order the operations would occur. Here are a few of the many possible:
Processing order: G1 gets the value from N 0 G2 Gets the value from N 0 that is, the first place in the thread counter is not directly operating memory G1 increase the value from 0 to 1 G1 1 writes to N G2 will increase the value from 0 to 1 G2 writes 1 to the N program output n value, currently 1
The name "Data race" is a bit of a misleading suspicion. Not only is the order of operations undefined, but there is no guarantee at all (no guarantees whatsoever). Compilers and hardware in order to achieve better performance, often the code in the upper and lower order of the internal and external transformation. If you see a thread in an intermediate behavior state, then the scene might be the same as shown in the following illustration:
The only way to avoid data contention is to synchronize access to all shared mutable data between threads. There are several ways to achieve this goal. In the go language, it is usually a pipe or a lock. (There are also lower-level mechanisms available in sync and sync/atomic packages, but not discussed in this article).
In the go language, the recommended way to handle concurrent data access is to use the pipeline to pass the actual data from one goroutine down to the next goroutine. The motto is good: " do not communicate through shared memory, but share memory through communication ." using local Variables
Func sharingiscaring () {
ch: = make (chan int)
go func () {
N: = 0//Only a goroutine visible local variable.
n++
CH <-N//data left from a goroutine ...
} ()
N: = <-ch //... Then safely reach another goroutine.
n++
FMT. PRINTLN (n)//output: 2
}The pipeline in the above code is doubly responsible-passing data from one goroutine to another goroutine and synchronizing: The sender Goroutine waits for another goroutine to receive the data, The receiver Goroutine also waits for another goroutine to send the data.
Go language memory model-to ensure that the value of reading a variable in one goroutine is exactly the value that is generated for the same variable write in another goroutine, the condition is quite complex, but the goroutine only shares all the mutable data through a pipe. Then you can stay away from the data competition . Mutual exclusion Lock
Sometimes it can be more convenient to synchronize data access by explicitly locking instead of using pipelines. The Go Language standard library provides a mutex lock-sync for this purpose. Mutex.
The key to this type of lock-in effect is that all access to shared data, whether read or written, can only be manipulated if Goroutine holds the lock . A goroutine error is enough to destroy a program and introduce data competition.
Therefore, a custom data structure should be designed with a clear API to ensure that all synchronizations are done within the data structure. In the following example, we built a secure, easy-to-use concurrency data structure, atomicint, for storing an integer value. Any number of goroutine can safely access this value through the Add and value methods.
If you have a problem, lock it and see who moves.
Atomicint is a concurrency data structure that holds an integer value
//The 0 value of the data structure is 0
type atomicint struct {
mu sync. Mutex//Lock, can only be held by one goroutine at a time.
n int
}
The Add method acts as an atomic operation to add N to Atomicint
func (a *atomicint) add (n int) {
a.mu.lock ()//waits for the lock to release and then holds it
A.N + = n
A.mu.unlock ()//release lock
}
//Value method returns the value of a
func (a *atomicint) value () int {
a.mu.lock ()
N: = a.n< C15/>a.mu.unlock ()//entire structure unlocked
return n
}
func Lockitup () {
wait: = Make (chan struct{})
var n Atomicint
go func () {
n.add (1)//Access
close (wait)
} ()
N.add (1)//Another concurrent access
<-wait
FMT. Println (N.value ())//output: 2
}Competition is sometimes very difficult to detect.
The function in the following example has a data contention problem, and the program will output 55555 when it executes.
Try it and maybe you'll get a different result.
sync. Waitgroup is part of the Go Language standard library and is used to wait for a set of Goroutine to finish running.
Func race () {
var wg sync. Waitgroup
WG. ADD (5)
///: Note The following line of code in the i++ for
i: = 0; i < 5; i++ {
go func () {
//Note what the next line of code will output. Why.
FMT. Print (i)//6 goroutine shared variable i
WG. Done ()
} ()
}
WG. Wait ()//Waits for All (5) Goroutine to run the end of
FMT. Println ()
}
Raceclosure.goFor output 55555, a plausible explanation is that the goroutine that executes i++ completes 5 i++ operations before the other Goroutine executes the print statement. The fact that the value of the variable i is updated is purely coincidental as seen by other goroutine.
A simple solution is to use a local variable and then pass the value as a parameter when the new Goroutine is opened: using local variables
Func correct () {
var wg sync. Waitgroup
WG. ADD (5) for
I: = 0; i < 5; i++ {
go func (n int) {//Use local variable
FMT. Print (n)
WG. Done ()
} (i)
}
WG. Wait ()
FMT. Println ()
}This time the code is right, the program will output the desired results, such as: 24031. Note: The order of operation between Goroutine is indeterminate.
Closures are still used, but it is possible to avoid data contention, and it is prudent to have each goroutine use a unique variable.
Func Alsocorrect () {
var wg sync. Waitgroup//Use Waitgroup Advanced Cargo
WG. ADD (5)
for I: = 0; i < 5; i++ {
N: = I//create a unique variable for each closure
go func () {
fmt. Print (n)
WG. Done ()
} ()
}
WG. Wait ()
FMT. Println ()
}In general, it is unlikely that you will be able to automatically detect all possible data contention, but go (starting with version 1.1) has a powerful data competition detector.
This tool is also easy to use: Just add the-race tag when you use the GO command. Opening the detector running the above program will give a clear and informative output:
$ go run-race raceclosure.go
race:
==================
warning:data race
Read by Goroutine 2:
Main.func 001 ().
/raceclosure.go:22 +0x65
Previous write by goroutine 0:
main.race ()
. /raceclosure.go:20 +0x19b
main.main ()
. /raceclosure.go:10 +0x29
runtime.main ()
. /go/src/pkg/runtime/proc.c:248 +0x91
goroutine 2 (running) created at:
main.race ()
. /raceclosure.go:24 +0x18b
main.main ()
. /raceclosure.go:10 +0x29
runtime.main ()
. /go/src/pkg/runtime/proc.c:248 +0x91
==================
55555
Correct:
01234
Also Correct:
01324
Found 1 Data race (s)
exit status 66The tool finds a competition for data that includes: A goroutine writes a variable on line 20th, and then another goroutine reads the same variable in the 22nd row without synchronizing.
Note: competition detectors can only find data that actually occurs during the run time competition (I don't quite understand that, please instruct) SELECT statement
The SELECT statement is the ultimate tool in the Go language concurrency toolset. Select is used to choose a further process from a set of possible communications. If any of the communication can be further processed, then randomly select one to execute the corresponding statement. Otherwise, if there is no default branch, the SELECT statement blocks until one of the communications is complete.
Here is an example of a toy that demonstrates how a SELECT statement can be used to implement a random number generator:
The Randombits function returns a pipe used to generate a bit random sequence of
func randombits () <-chan int {
ch: = make (chan int)
go func () {
fo R {
Select {case
ch <-0://NOTE: The branch does not have a corresponding processing statement case
CH <-1:
}}
} ()
return CH
}Here's a more practical example: how to set a time limit for an action using the SELECT statement. The code outputs the value of the variable news or the timeout message, depending on which of the two receive statements is executed first:
Select {Case
news: = <-newsagency:
fmt. PRINTLN (News) Case
<-time. After (time. Minute):
FMT. Println ("Time Out:no news in one minute.")
}function time. After is part of the Go Language standard library, it sends the current time to the returned pipeline after waiting for a specified time.
Synthesize all examples
Take some time to study this example carefully. If you fully understand it, you will have a complete grasp of the concurrency in the Go language.
This program demonstrates how to send and receive data by any number of goroutine, and also demonstrates how to use a SELECT statement to select one from multiple communications.
Func Main () {
people: = []string{"Anna", "Bob", "Cody", "Dave", "Eva"}
match: = Make (chan string, 1)//For an unmatched send operator To provide a space
WG: = new (sync. Waitgroup)
WG. ADD (len (People))
for _, Name: = Range people {
go Seek (name, match, WG)
}
WG. Wait ()
Select {Case
name: = <-match:
fmt. Printf ("No one received%s ' s message.\n", name)
default:
//No pending send Operation
}
}
//function seek Send a name to the match pipeline or receive a peer from the match pipeline, and at the end of the notification wait group
func Seek (name string, Match Chan string, wg *sync. Waitgroup) {
Select {case
peer: = <-match:
fmt. Printf ("%s sent a message to%s.\n", peer, name) case
match <-name:
//wait for a goroutine to receive my message
}
WG. Done ()
}Example output:
Parallel Computing$ go Run matching.go Cody sent a message to Bob. Anna sent a message to Eva. No one received Dave ' s message.
One application of concurrency is to divide a large computation into a number of units of work that are scheduled to be computed simultaneously on different CPUs.
Distributing computations to multiple CPUs is more an art than a science. Here are some rules of thumb: each unit of work should take about 100 microseconds to 1 milliseconds to compute. If the unit granularity is too small, the slicing problem and the management overhead of scheduling sub-problems may be too large. If the unit granularity is too large, the entire calculation may have to wait for a slow work item to end. This slowness can occur for a variety of reasons, such as scheduling, interruption of other processes, or poor memory layout. (Note: The number of units of work is not dependent on the number of CPUs) to minimize the amount of data that is shared. Concurrent writes are expensive, especially if the goroutine is running on a different CPU. Data sharing between read operations is usually not a problem. Data access uses good locality as much as possible. If the data is kept in the cache, the data is loaded and stored much faster, which is especially important for write operations.
The following example shows how to slice a computationally expensive calculation and distribute it across all available CPUs. Let's look at the code that needs to be optimized:
Type Vector []float64
//function convolve Calculation w = u * V, where w[k] =σu[i]*v[j], i + j = k
//Prerequisites: Len (U) > 0, len (v) &G T 0
func convolve (u, v vector) (w vector) {
N: = Len (u) + len (v)-1
w = make (vectors, n) for
k: = 0; K < N k++ {
W[k] = mul (U, V, k)
}
return
}
//function Mul return σu[i]*v[j], i + j = k.
Func mul (U, v Vector, K int) (res float64) {
N: = Min (k+1, Len (U))
J: = min (k, Len (v)-1) for
i: = k-j; I < n; I, j = i+1, j-1 {
res + = u[i] * v[j]
}
return
}The idea is simple: determine the appropriate size of the unit of work, and then execute each unit of work in different goroutine. The following is a concurrent version of Convolve:
Func convolve (U, v vector) (w vector) {
N: = Len (u) + len (v)-1
w = make (Vector, N)
//W cut into cost ~100μs-1ms To the calculated work cell
Size: = Max (1, 1<<20/n)
WG: = new (sync. Waitgroup)
WG. ADD (1 + (n-1)/size) for
I: = 0; i < n && i >= 0; i + = size {//integer overflow after i < 0
J: = i + size
if J > N | | J < 0 {//integer overflow J < 0
j = N
}
//These goroutine shared memory, but read-only
go func (i, J int) {for
k: = i; k < ; J k++ {
W[k] = mul (U, V, k)
}
WG. Done ()
} (i, j)
}
WG. Wait ()
return
}After the work cell is defined, it is usually better to hand over the dispatch to the runtime and to the operating system. However, for Go 1.* you may need to tell the runtime how many goroutine you want to run the code at the same time.
Func init () {
numcpu: = runtime. NUMCPU ()
runtime. Gomaxprocs (NUMCPU)//try to use all available CPUs
}Additional: Understanding Golang Concurrency Principle
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
