"Translate" concurrency, co-processes and Gomaxprocs

This is a creation in Article, where the information may have evolved or changed.

# # #介绍

When newcomers join the Go-miami group, they often want to learn the concurrency model. When I first started to hear about the go language, concurrency seemed to be a buzzword in the language. When I saw the video of Rob Pike's go concurrency mode, I thought I should learn the language.

In order to understand how to write more simple and error-prone concurrent programs through the Go language, we first need to understand what the results of concurrent programs and concurrent programs are. I'm not going to tell the CSP (communication sequence process, communicating sequential Processes), which is the basis of the Go Language implementation pipeline (channel). This article will focus on what concurrency is, what the role of the Gomaxprocs, how environment variables and run-time functions affect the execution of the Go Language program.

# # #进程和线程

When we run an application, such as the one I used to write this article, the operating system creates a process for the app. The task of a process is like a container that uses and manages resources for this application to run. These resources include the memory address space, the handle to the file, the device, and the thread.

A thread is a part of execution that is dispatched by the operating system to execute the code inside the process that we implement in the function. A process starts with a thread, also called the main thread, and ends when the thread ends the entire process. This is because the main thread is the source of the application. The main thread can in turn create more threads, and these threads can also create more threads.

The operating system can dispatch a thread to execute on one of the available processors, regardless of the processor on which the parent thread handles the thread. Each operating system has its own scheduling algorithm, not a specific one, to determine the best way for us to develop parallel programs. And these algorithms will change with each upgrade of the operating system, this needs our attention.

# # #协程和并行

Any function or method inside the go language can create a process. We can assume that the main method execution is a co-process, but the Go runtime does not start the process. The process can be considered lightweight because they consume very little memory and resources, and it initializes a small stack space. The previous Go language 1.2 version stack space will be initialized to 4 K, and the current 1.4 version will be initialized to 8K. This stack can automatically add space as needed.

The operating system schedules the available processors to execute threads, and the Go runtime dispatches the process through an operating system thread. By default, the go runtime allocates a logical processor to execute all the threads defined in the code. Even with this logical processor and operating system thread, hundreds of processes can be executed efficiently and quickly by concurrent calls. Adding more logical processors is not recommended, but if you want to run the coprocessor in parallel, go provides an addition to the logical processor by GOMAXPROCS environment variables or run-time functions.

concurrency is not parallel. Parallelism is the simultaneous execution of two or more than two threads at the core of a multicore processor that is not in parallel. If you configure more than 1 logical processors at run time, the scheduler will allocate the coprocessor on top of these logical processors, and the result is that multiple threads are executed on different operating system thread. However, to really implement parallel execution your server must be a multi-core processor. If not, even if the go runtime is set to require more than one logical processor, the coprocessor will still execute the co-process concurrently on the same processing processor.

# # #并发的例子

Let's create a small program to show the go concurrency execution coprocessor S. This example is done under the default setting, which is to use a logical processor.

package mainimport (    "fmt"    "sync")func main() {    var wg sync.WaitGroup    wg.Add(2)    fmt.Println("Starting Go Routines")    go func() {defer wg.Done()for char := 'a'; char < 'a'+26; char++ {    fmt.Printf("%c ", char)}    }()    go func() {defer wg.Done()for number := 1; number < 27; number++ {    fmt.Printf("%d ", number)}    }()    fmt.Println("Waiting To Finish")    wg.Wait()    fmt.Println("\nTerminating Program")}

This program go launches two threads through the keyword, declaring two anonymous functions. The first process prints the lowercase English alphabet, and the second prints 1 to 26 of these numbers. When we run this program, we will get the following results:

Starting Go RoutinesWaiting To Finisha b c d e f g h i j k l m n o p q r s t u v w x y z 1 2 3 4 5 6 7 8 9 10 1112 13 14 15 16 17 18 19 20 21 22 23 24 25 26Terminating Program

Let's look at the result of the output and the code is executed concurrently. Once the two processes are started, the master's process will wait for both of the execution to finish. We need to use this method otherwise the entire program ends when the main process runs. WaitGroupis a nice towners used to communicate whether or not the communication is done between different co-processes.

We can see that the first process shows a total of 26 letters, and the second one starts showing 26 numbers. Because the two processes are executing too fast, and the execution ends in milliseconds, we are not able to see if the scheduler has interrupted it before the first coprocessor has finished executing. We can add a wait time in the first process to judge the scheduler's strategy.

package mainimport (    "fmt"    "sync"    "time")func main() {    var wg sync.WaitGroup    wg.Add(2)    fmt.Println("Starting Go Routines")    go func() {defer wg.Done()time.Sleep(1 * time.Microsecond)for char := 'a'; char < 'a'+26; char++ {    fmt.Printf("%c ", char)}    }()    go func() {defer wg.Done()for number := 1; number < 27; number++ {    fmt.Printf("%d ", number)}    }()    fmt.Println("Waiting To Finish")    wg.Wait()    fmt.Println("\nTerminating Program")}

This time we added 1 seconds of wait time in the first process, and the call sleep caused the scheduler to exchange the execution order of the two threads:

Starting Go RoutinesWaiting To Finish1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 ab c d e f g h i j k l m n o p q r s t u v w x y zTerminating Program

This number precedes the letter. sleepaffects the scheduler stopping the first process, first executing the second one.

# # #并行的例子

The two examples above are all concurrent execution, not parallel. Let's change the code to allow the code to execute in parallel. All we need to do is add a logical processor to the scheduler so that he can use two threads:

package mainimport (    "fmt"    "runtime"    "sync")func main() {    runtime.GOMAXPROCS(2)    var wg sync.WaitGroup    wg.Add(2)    fmt.Println("Starting Go Routines")    go func() {defer wg.Done()for char := 'a'; char < 'a'+26; char++ {    fmt.Printf("%c ", char)}    }()    go func() {defer wg.Done()for number := 1; number < 27; number++ {    fmt.Printf("%d ", number)}    }()    fmt.Println("Waiting To Finish")    wg.Wait()    fmt.Println("\nTerminating Program")}

This is the output of the program:

Starting Go RoutinesWaiting To Finisha b 1 2 3 4 c d e f 5 g h 6 i 7 j 8 k 9 10 11 12 l m n o p q 13 r s 14t 15 u v 16 w 17 x y 18 z 19 20 21 22 23 24 25 26Terminating Program

Every time we run this program we will get different results. The scheduler does not perform the same results every time. We can see that the co-process is actually executed in parallel this time. Both processes start running immediately, and you can see that two of them are competing to output their respective results.

# # #结论

We can add multiple logical processors to the scheduler, but that doesn't mean we have to do this. The Go team has a reason to set the run-time parallel number to 1 by default. Arbitrarily adding logical processors and parallel threads does not necessarily provide better performance for your program. To do a performance stress test, make sure that when you modify the Go runtime, Gomaxprocs must be able to optimize performance.

Creating concurrent co-processes within our application will eventually cause our process to attempt to access the same resources at the same time. Read and write shared resources must be atomic. In other words, the read and write operations must be executed in the same moment in a single process, otherwise we need to create a critical condition in the code. Learn more about the competitive conditions you can read in this article.

Pipelines are a safe and elegant way to write concurrent programs inside the go language, eliminating the requirement to write competitive conditions that make developing parallel programs more interesting. We now know how the process works, is dispatched, and if it is executed in parallel, then we will talk about pipelines.

###### Reference 1. Concurrency, Goroutines and GOMAXPROCS2. The difference between concurrency and parallelism: the analogy of eating steamed bread