The beauty of Go language concurrency

EMC China Research Institute Yenkai
Brief introduction
multi-core processors are becoming more and more popular, there is no easy way to let us write software to release the power of multi-core. The answer is: Yes. With the rise of Golang, Erlang, scale and other programming languages, new concurrency patterns are becoming clearer. Just as procedural programming and object-oriented, a good programming model needs to have a very concise kernel, as well as a rich extension on this, can solve the real world of various problems. In this paper, go language is used as an example to explain the kernel and extension.
The kernel of the concurrency pattern
The kernel of this concurrency pattern requires only the coprocessor and the channels. Where the coprocessor is responsible for executing code, the channel is responsible for passing events between the threads.
Concurrent programming has always been a very difficult task. To write a good concurrency program, we have to understand how threads, locks, semaphore,barrier, and even CPUs update the cache, and that they all have a strange temper, a trap everywhere. I will never operate these underlying concurrency elements myself unless I can. A simple concurrency model does not require these complex underlying elements, only the process and channels are sufficient.
A coprocessor is a lightweight thread. In procedural programming, when a procedure is invoked, it needs to wait for its execution to return. When you call a coprocessor, you do not have to wait for it to finish and return immediately. The process is very lightweight, and the go language can perform up to 100,000 of the processes in one session, and still remain high-performance. For a common platform, a process has thousands of threads, and its CPU is busy with context switching, and performance drops dramatically. Creating threads at random is not a good idea, but we can use a lot of the coprocessor.

Channels are data transfer channels between the threads. Channels can pass data between many of the threads, and can be a reference or a value. There are two ways of using a channel.

· The coprocessor can attempt to put data into the channel, and if the channel is full, the coprocessor hangs until the channel can put data into it.

· The coprocessor can attempt to request data from the channel, and if the channel has no data, it suspends the coprocessor until the channel returns the data.

Thus, the channel can control the running of the coprocessor while passing the data. A bit like event-driven, it's kind of like blocking queues. These two concepts are very simple, each language platform will have a corresponding implementation. There are also libraries on both Java and C that can implement both.

As long as there are threads and channels, you can gracefully solve the problem of concurrency. You do not have to use other and concurrency-related concepts. How to use these two blades to solve all kinds of practical problems.
The extension of concurrent mode
The threads relative can be created in large quantities more than threads. Open this door, we develop new usage, can do generator, can let function return "service", can let loop execute concurrently, also can share variable. But with new usage, there are new tricky issues, and the process can leak and improper use can affect performance. Various usages and issues are described below. The demo code is written in the go language because it is simple and straightforward, and supports full functionality.
Build device
Sometimes we need to have a function that keeps generating data. For example, this function can read a file, read a network, generate a growth sequence, and generate random numbers. These behaviors are characterized by the known variables of the function, such as the file path. And then keep calling to return the new data.

Let's take a random number for example so that we can do a random number generator that executes concurrently. The non concurrency approach is this:

function rand_generator_1, returning int

funcrand_generator_1 () int {

Return to Rand. Int ()}
Above is a function that returns an int. If Rand. Int () This function call takes a long time to wait, and the caller of the function hangs. So we can create a coprocessor that specifically performs Rand. Int ().

function rand_generator_2, return channel (Channel)

Funcrand_generator_2 () Chan int {

Create a Channel

Out: = Make (chan int)

Create a co-process

Go func () {

for {

Writes data to the channel, if no one reads will wait

Out <-Rand. Int ()

}

}()

Return out

}

Funcmain () {

Generate random numbers as a service

Rand_service_handler: =rand_generator_2 ()

Read random numbers from the service and print

Fmt. Printf ("%d\n", <-rand_service_handler)

}

This section of the above function can concurrently execute rand. Int (). It is worth noting that the return of a function can be understood as a "service". But we need to get random data, we can always access to this service, he has prepared the appropriate data for us, no need to wait, with. If we call this service not very often, a coprocessor is enough to meet our needs. But what if we need a lot of access.         We can use the multiplexing technology described below to start a number of generators, and then integrate them into a large service. Call generator, you can return a "service". Can be used in the context of continuous data acquisition. Uses a wide range, reads data, generates IDs, and even timers. This is a very concise way of thinking, the process of concurrency.

Multi-channel multiplexing

Multiplexing is the technology that lets you handle multiple queues at once. Apache requires a process to process each connection, so its concurrency performance is not very good. and Nginx use multiplexing technology, so that a process to handle multiple connections, so concurrency performance is better. In the same way, multiplexing is also needed, but it is different in the context of a synergistic process. Multiplexing can combine several similar small services into one large service.

So let's use multiplexing to make a higher concurrent random number generator.

function Rand_generator_3, return channel (Channel)

Funcrand_generator_3 () Chan int {

Create two random number generator services

rand_generator_1: = rand_generator_2 ()

Rand_generator_2: = rand_generator_2 ()

Create a Channel

Out: = Make (chan int)

Create a co-process

Go func () {

for {

Read the data in Generator 1 to consolidate

Out <-<-rand_generator_1

}

}()

Go func () {

for {

Read the data in Generator 2 to consolidate

Out <-<-rand_generator_2

}

}()

Return out}

Above is a high concurrent version of the random number generator using multiplexing technology. By consolidating two random number generators, this version is twice times more capable. Although the coprocessor can be created in large quantities, many of the threads still scramble for the output channel. The Go language offers a select keyword to solve, and each home has its own tips.         Increasing the buffer size of the output channel is a common solution. Multiplexing technology can be used to consolidate multiple channels. Improve performance and ease of operation. The use of other modes has great power.

Future technology

Future is a very useful technique, and we often use future to manipulate threads.  When using threads, we can create a thread, return to future, and then wait for the result to go through it. But in the future environment can be more thorough, input parameters can also be future.

When calling a function, it is often the argument is ready. The same is true when calling a coprocessor. But if we set the incoming argument to a channel, we can call the function without the parameters ready. Such a design can provide a great degree of freedom and concurrency. The function call and the function parameter preparation process can be completely decoupled. Here's an example of accessing a database using this technique.

A query structure body

typequery struct {

Parameter Channel

SQL Chan string

Results Channel

Result Chan string

}

Execute Query

Funcexecquery (q query) {

Start the coprocessor

Go func () {

Get input

sql: = <-q.sql

Accessing the database, outputting the result channel

Q.result <-"get" + SQL

}()

}

Funcmain () {

Initialize Query

Q: =

Query{make (Chan string, 1), make (Chan string, 1)}

Execute query and note that you do not need to prepare parameters for execution

ExecQuery (q)