Ready? Go! Next article: Start with multiple cores

Google released the go programming language in November 2009, aiming to have both the efficiency of C language and the simplicity of Python. In March this year, the Go development team officially released go 1, the first stable release of the Go language. This means that the go language and its standard library have been stabilized, and developers can now use it as a stable development platform to build their own applications. We will use two articles to introduce the features and application of the Go language. This article is the second one.

The previous section introduced some of the Go language syntax and type systems. This article focuses on the Parallel Processing and go tool chain of the Go language. The complete code in this article can be found on GitHub.

Parallel and goroutine

However, the development of processor technology points out that compared to [covering a variety of parallel structures] single processor, multiple similar processors (each containing its own storage unit) A multi-processor computer may be more powerful, reliable, and cost-effective.--- C. a.r. Hoare, winner of the Tuling Award, CSP author, in December 1978

In the 60 or 70 s, parallel computing once became a research hotspot to make up for the processing capability of the processor. There are no lack of good ideas during this period, such as semaphores (semaphore), processes (monitors), locks (mutex), and message transmission-based synchronization mechanisms. However, since 1980s, with the rapid improvement of Single-core processor performance, the academic community has ushered in the dark era of parallel computing. Among the research results of the 60 or 70 s, only some early ideas were widely used in actual development. However, many achievements in the late 1970s s have not even been applied in a large scale, and they are accompanied by the arrival of the Black-dark period of parallel computing, or they are not warm or collected to the database. CSP (Communicating Sequential Processes) is one of them. However, its elegant and concise processing method is still circulating in some small languages. Nowadays, due to energy consumption and heat dissipation, the development of processors has switched to multi-core approaches to improve processor performance. We once again ushered in parallel computing that we once faced. At this time, the CSP model is gradually exposed.

The basic idea of CSP is message-based synchronization and data sharing. Unlike traditional lock synchronization, the message mechanism simplifies program design and effectively reduces potential bugs. The CSP-based language has three main branches: one that is loyal to the original CSP design and represented by OCCAM; the other that emphasizes the network and mode, represented by Erlang; another is a channel that emphasizes message transmission, represented by Squeak, newsqueak, Alef, limbo, and go. It is worth mentioning that most of the third language is hosted by Rob Pike or participated in development, which naturally includes go.

Since this branch of Go is characterized by a channel, it begins with the go channel. The go channel is a data type that can be used by goroutine to transmit data. We will discuss in detail what a goroutine is. Here, you only need to understand it as a runtime structure similar to the thread.

To define a channel, you must specify the Data Type transmitted over this channel. It can be int, float32, float64 and other basic data types, user-defined struct, interface, or even channel itself.

ch := make(chan int)

In this way, a channel that transmits an integer type is defined. If you want to read a value from this channel, you can use the <-operation. Similarly, use the-> operator for writing:

// Read a value from CH and store it in I: = <-ch // write the value ch of J to Ch <-J

Channel operations are synchronous. A read operation continues to execute the following statement only after the actual content is read. Write operations are read only when data is written to the other end of the channel, after the statement is executed. (Go CITIC channels can also be added to cache queues, which will not be discussed here)

At the same time, for the channel, you can also use the for loop to process the content from the channel in sequence:

func handle(queue chan *Request) { for r := range queue { process(r) } }

The task of this function is to constantly read the pointer of the Request struct from the channel, and then call the process function for processing.

In addition, you can use select to perform read and write operations on multiple channels:

func Serve(queue chan *Request, quit chan bool) { for { select { case req := <- queue: process(r) case <- quit: return } } }

This function takes two channels as parameters. The first channel queue is used to transmit various requests. The second channel, quit, is used to publish a signaling message, telling the function to return.

Next we will talk about goroutine. It is a lightweight parallel structure than a thread. When the go program runs, it generally runs several threads in parallel and then assigns the goroutine to each thread. When a goroutine ends or is blocked, the other goroutine will be scheduled to the thread where the blocked or terminated goroutine is located. Such scheduling ensures that each thread can have a high usage and does not have to be stuck. This saves many Operating System Scheduling threads and causes context switching. According to the official go statement, it is quite normal for a go program to run tens of thousands to hundreds of thousands of goroutines at the same time.

Using a goroutine is also very simple, as long as you add go before the function call:

go process(r)

In this way, the process function runs in a goroutine.

The result is that the processing of concurrent connections on the server is greatly simplified. As we all know, if a thread processes only one user connection, it will be very simple to develop, but the efficiency is not high. If a thread processes multiple user connections, it makes development more difficult. The use of goroutine in combination with the channel not only increases the development difficulty, but also increases the efficiency.

Consider an application scenario where the server receives client requests from the network, performs some processing, and then returns the results to the customer.

Use different goroutines for different user connections. Define a struct named userconn to indicate a user connection. At the same time, this struct defines a method called readrequest, which is used to read user requests from the network; there is also a method called writeresponse, which is used to deliver results to users from the network. As an imaginary example, the specific implementation details are not described here.

So, for each connection, we need to do something like this:

Func serveclient (conn * userconn) {CH: = make (chan * response) // create a goroutine, // specifically used to send the result to the user go writeres (Conn, CH) for {// read a request, // judgment type // if the user request is closed, // The function returns Req: = Conn. readrequest () Switch req. type {Case normal_request: Go process (req, CH) Case Exit: return }}}

The basic structure of writeres and process is as follows:

func writeRes(conn *UserConn, ch chan *Response) { for r := range ch { conn.WriteResponse(r) } } func process(req *Request, ch chan *Response) { res := calculate(req) ch <-res }

The channel itself is in line with people's intuitive definitions of communication tools. developers can naturally use the channel to establish various relationships between goroutines. With channels and goroutine, the tasks to be completed by each function are single, reducing the possibility of errors. In the code, the memory space is shared by passing pointers. messages are synchronized before each sharing. This is another go principle: the memory is shared by passing messages, rather than using the shared memory to transmit messages. This simplifies the development of parallel programs.

As a practical programming language, go does not follow the CSP's original paper and only provides channel methods for synchronization. Go also provides lock-based, semaphore-based and other traditional synchronous machine tools in the standard library. In the above Code, there is actually a potential BUG: The serveclient function does not exit after all goroutine running the process is executed, however, the client directly exits after receiving the exit command from the client. More Reasonable operations should be returned after all the goroutine processing the connection exits. In the standard library, a waitgroup struct can specifically solve the problem of waiting for multiple goroutines. This is not detailed here.

Next, enable a goroutine for each user connection and execute the serveclient function. As mentioned above, because goroutine is a scheduling unit that is more lightweight than a thread, such a number of goroutine will not cause serious performance degradation.

Since the goroutine and message mechanism simplify development and go encourages such design, developers will consciously choose a design based on multiple goroutine. Another benefit of this is the scalability of applications on multi-core systems. With the increase in the number of processor cores, it is a great challenge for developers to explore the internal parallel structure of the program. However, when writing with go, the Design Based on Multiple goroutines usually has enough parallel structures to be extended to the multi-core processor. Each goroutine can be placed on an independent processor and executed in parallel with other goroutine. That is to say, the code written for the quad-core processor today may run on the 128-core CPU and use all the cores at the same time without modification.

Directly compile without Configuration

If go needs a configuration file to describe how to compile and build a program written in go, it is the failure of go.--- Go official documentation

For tools such as make, Autoconf, and automake used to specify the compilation sequence and dependency, the go attitude is: when developers write code, they leave enough information about dependencies, developers should not be required to write a configuration file separately to specify the dependency and compilation sequence. Therefore, developers only need to configure a directory structure and an environment variable according to the official documents after installing the go tool chain. In the future, only a few simple commands are required to compile any go Program/library.

For a self-contained Program (independent from any third-party library), you only need to run go build in the current directory to compile the entire program.

What if my program depends on a third-party library? It's easy to write the third-party library's position in the network in the Import Statement in the code. Here, the import concept is the same as that in Java/python, And a package is introduced.

import ( "fmt" "github.com/monnand/goredis" )

The first package introduced in import is FMT, which is a package in the standard library and provides formatting input and output for the printf class. The second introduced package is the code base on GitHub. It will introduce the goredis project-defined package under monnand on GitHub.

Next, call the go command to install the Library:

go get github.com/monnand/goredis

In this way, the Go program automatically downloads, compiles, and installs the Library (including its Dependencies ). Then, use go build to compile a program dependent on goredis.

In addition, if goredis-dependent programs are in GitHub (or other go-supported version control libraries, it is enough to specify the remote address of the program with only one go get command. Go downloads and installs various Dependencies by itself. In addition to GitHub, go also supports Google Code, Bitbucket, Launchpad, or any go programs/libraries on other servers that use SVN, git, bazzar, and mercurial for version control. This greatly simplifies the operations of developers and end users.

Talking about Operation Efficiency

• MATT: after Pat/Go is used, how much is the efficiency of the json api node improved compared to the (original) Sinatra/Ruby solution? Just give an estimate.
• Blake: approximately 10,000 times
• MATT: Pretty! Can I quote your words?
• Blake: I will check it again. I think it seems I have underestimated it.

--- Conversations between Matt aimonetti and Blake mizerany on Twitter.

The efficiency of the Go program has always been the focus of attention. On the one hand, go syntax and type systems are very simple, providing a lot of space for Compiler development and optimization. On the other hand, as a statically compiled language, go directly compiles code into machine code without intermediate interpretation.

However, if you search for it on the internet, you will find that there is a serious polarization in the efficiency of the Go program. Some tests show that the running efficiency of Go programs is very high, and even some aspects exceed the same program written in C ++. Another part of the test is practical. In some aspects, go is not even as good as the script written by stackless python.

Although the go compiler has a lot of optimization space, the efficiency of the generated machine code is relatively high. The standard library, including various runtime codes, such as garbage collection and hash tables, is not optimized yet, and some are still in a very preliminary stage. This is one of the reasons why the test results on the network are significantly different. In addition, as a new language, developers are not familiar with it, and the written code may have performance bottlenecks, which also increases the differences in the evaluation results.

Russ Cox, a developer of the Go language, published an article on the go official blog. The code of a benchmark test program (benchmark) is used, and the C ++ test and go test sections are optimized respectively. The optimized go program running time is only 65.8% of the optimized C ++ program running time! This also reflects the potential of go from one side.

At present, there are still many defects in the go language: Garbage collection is still in the initial stage, and the support for 32-bit systems is not complete yet. The code of some standard libraries still needs to be optimized. According to the official go statement, a completely parallel garbage collector will be used in the future, which will greatly improve the performance. With the release of go 1, go developers will also shift their focus from the syntax and standard library specifications to the optimization of compilers and standard libraries. The goal of running the go program is to approach C ++ and surpass Java.

Summary

Now, I think it is much better than C ++ in terms of system-level development. It is more efficient for development and can be used to solve many problems in a simpler way than C ++.---- Bruce Eckel, author of C ++ programming ideology and Java programming ideology

Ken thonpson, founder of UNIX, Rob Pike, Russ Cox, developer of UNIX/Plan 9, Brad Fitzpatrick, memcached author, one of Java hotspot compilers, Robert griesemer, one of chrome V8 engine authors, and gold connector authors, ian lancetaylor, an active developer in the GCC community ...... When such a group of people get together, no matter what the development, the team itself may be enough to attract everyone's attention. As a language developed by such a team, go has brought many surprises so far.

Many companies already use go to develop production-level programs. Rob Pike once revealed that Google is gradually using go. YouTube uses go to write Core Components and organizes some code into the open-source project vitess. Chinese companies, including Douban and QBOX, have also taken the lead in the go language field.

With the launch of go 1, a stable go language platform and open-source community have been formed. Go is a choice for developers who like to try new languages.

Ready? Go! Next article: Start with multiple cores