Translation Go's Race detector

This is a creation in Article, where the information may have evolved or changed.

Understanding the race is important for concurrent programming, and it is also a very effective optimization method if you can understand the race state in the program by some means, so as to further adjust to avoid the race state. The Go 1.1 tool chain introduces a race detector to detect and present race conditions in the program. The Go team wrote a blog post detailing the rationale and use of the tool. The original is in this "introducing the Go Race Detector".

———— Translation Divider Line ————

Go's Race detector

Dmitry Vyukov and Andrew Gerrand

Race condition is almost the most concealed and difficult to find program error. They often lead to strange and unexplained errors, especially after the code has been deployed to the production environment for a long time. While the concurrency mechanism of Go makes it easy to write clear concurrent code, they cannot avoid race conditions. Careful and diligent testing is a must. and tools can help.

We are pleased to announce that Go 1.1 includes a race detector, a new tool for discovering race conditions in the go code. It is currently available in Windows systems with 64-bit x86 processors for Linux, MacOS, and 64-bit x86 processors .

The race detector is based on the C + + Threadsanitizer Runtime Library, which has been used in Google's base code and Chromium, and has detected many errors. The technology was integrated into go in December 2012, and since then it has detected 42 race codes in the standard library. It is now part of the continuous integration process and will continue to discover race conditions and capture.

Working principle

This race detector is integrated into the Go tool chain. When the command-line parameter-race is set, the compiler logs all memory accesses in the code, including when and how it is accessed, while the runtime library monitors the unsynchronized shared variables. When this "dirty" behavior is detected, a warning is printed. (see this article for details about the algorithm.) ）

Because of its design, this race detector can only detect race conditions that are triggered by code that is running, which means that it is important to let the execution of the open race run under real working pressure. However, it is unrealistic to have a race detector enabled by using 10 times times the CPU and memory to open a competing execution file. One way to avoid this dilemma is to run some tests with the race detector turned on. Because stress tests and integration tests tend to be more likely to validate concurrent parts, they are a good choice. Another way to use a production environment workload is to deploy an open-race instance into a service pool (tcpcopy may also be a good choice).

The use of a race-state detector

The race detector has been fully integrated into the Go tool chain. In order to compile code that opens the race detector, you only need to add the command line parameter-race ID:

If you want to try out a race detector yourself, get and execute this example program:

$ go get-race code.google.com/p/go.blog/support/racy$ racy

Instance

There are two real cases captured by a race detector.

Example 1:timer.reset

The first simple example is a real error found by the race detector. It uses a timer to print a message after a random 0-1-second delay. Then repeat this process for five seconds. It uses time. Afterfunc creates a timer for the first message, and then uses the Reset method to dispatch the next message, reusing the timer every time.

Package Mainimport (        "FMT"        "Math/rand" "Time        ") func main () {    Start: = time. Now ()    var t *time. Timer    t = time. Afterfunc (Randomduration (), func () {        fmt. Println (time. Now (). Sub (start))        T.reset (Randomduration ())    }) time    . Sleep (5 * time. Second)}func randomduration () time. Duration {    return time. Duration (Rand. Int63n (1E9))}

This may seem like a reasonable code, but in some specific circumstances it will go wrong in a singular way:

Panic:runtime Error:invalid memory address or nil pointer dereference
[Signal 0xb code=0x1 addr=0x8 pc=0x41e38a]

Goroutine 4 [Running]:
Time.stoptimer (0x8, 0x12fe6b35d9472d96)
Src/pkg/runtime/ztime_linux_amd64.c:35 +0x25
Time. (*timer). Reset (0x0, 0x4e5904f, 0x1)
src/pkg/time/sleep.go:81 +0x42
Main.func 001 ()
Race.go:14 +0xe3
Created by Time.gofunc
src/pkg/time/sleep.go:122 +0x48

What's going on here? Open Race Detector Running this program will be clearer:

==================
Warning:data RACE
Read by Goroutine 5:
Main.func 001 ()
Race.go:14 +0x169

Previous Write by Goroutine 1:
Main.main ()
Race.go:15 +0x174

Goroutine 5 (running) created at:
Time.gofunc ()
src/pkg/time/sleep.go:122 +0x56
Timerproc ()
src/pkg/runtime/ztime_linux_amd64.c:181 +0x189
==================

The race detector shows the problem: Unsynchronized reads and writes from different goroutine to the variable T. If the delay inside the timer is small, the timer function may be executed before the main goroutine is assigned to the variable T, while the t.reset is raised on T with a value of nil.

To fix this race condition, you only need to modify the code that the main goroutine reads and writes to the variable t:

Func Main () {     Start: = time. Now ()     Reset: = Make (chan bool)     var t *time. Timer     t = time. Afterfunc (Randomduration (), func () {         fmt. Println (time. Now (). Sub (start))         reset <-True     }) for time     . Since (Start) < 5*time. Second {         <-reset         t.reset (randomduration ())     }}

The main goroutine here is fully responsible for setting and resetting the timer T, while the newly added channel reset is used to communicate to ensure that the timer is reset in a way that is safe for the thread.

A simple and more efficient approach is to avoid reusing timers.

Instance 2:ioutil. Discard

The second example is more subtle.

The Discard object of the Ioutil package implements IO. Writer, to discard all the data written to it. It can be likened to/dev/null: a place where you can send data without storing them. This can usually be used in IO. Copy empties a Reader, just like this:

Io. Copy (Ioutil. Discard, Reader)

Back in July 2011, the go team noticed this approach to using Discard inefficient: The Copy function allocates a buffer of up to a kB at the time of each invocation, but when the Discard is used, the buffer is not necessary as long as the read data is discarded. We do not think there should be such a significant overhead for this customary usage of Copy and Discard.

The way to fix it is simple. If the provided Writer implements the Readfrom method, Copy will invoke it like this:

Io. Copy (writer, reader)

This is a delegate that implies a more efficient invocation:

Writer. Readfrom (reader)

The Readfrom method is added to the underlying type of Discard so that the internal buffer is shared among all its users. We know that in theory this is a race condition, but since all the data written in the buffer are discarded, it is not considered important.

When the race detector is implemented, it immediately identifies a problem with the code. Again, we don't think that this code will have any problem, and that the race condition is not "real". To avoid "pseudo-errors" in the build, we implemented a version that was not race-only and worked only when the race detector was turned on.

But a few months later, Brad came across a frustrating and strange mistake. After several days of commissioning, the final position was due to ioutil. Discard caused by the real race condition.

Here is the code known as race in Io/ioutil, Discard is devnull and shares the same buffer across all users.

var blackhole [4096]byte//Shared buffer func (devnull) readfrom (R io. Reader) (n Int64, err error) {    readsize: = 0    for {        readsize, err = R.read (blackhole[:])        n + = Int64 (readsiz e)        If err! = Nil {            If err = = Io. EOF {                return n, Nil            }            return        }}    }

Brad's program contains a trackdigestreader type that encapsulates IO. Reader and logs a hash check of the information read.

Type Trackdigestreader struct {    R io. Reader    H Hash. Hash}func (t Trackdigestreader) Read (P []byte) (n int, err error) {    N, err = T.r.read (p)    t.h.write (P[:n])    R Eturn}

For example, you can use it to compute the SHA-1 hash value while reading a file:

TDR: = Trackdigestreader{r:file, h:sha1. New ()}io. Copy (writer, TDR) fmt. Printf ("File Hash:%x", tdr.h.sum (nil))

In some cases there is no place to write this data, but you still need to get a hash of the file, so you can use Discard:

Io. Copy (Ioutil. Discard, TDR)

But in this case, the blackhole buffer is not a black hole; it is a save from source IO. Reader to read the data, and then write it to the hash. A proper place for the Hash. When multiple goroutine simultaneously hash a file, all share the same blackhole buffer, and the race condition proves its existence again by messing up the data between the read and hash. There is no error or panic occurs, but the hash is wrong. It's awful!

Func (t Trackdigestreader) Read (P []byte) (n int, err error) {    //the buffer p is blackhole    n, err = T.r.read (p) 
  //p May is corrupted by another goroutine here,    //between the Read above and the Write below    t.h.write (P[:n])    return}

By giving each ioutil. The Discard provides a separate buffer, eliminating the race condition caused by the shared buffer, which is eventually fixed.

Summarize

A race detector is a powerful tool for checking the correctness of concurrent programs. It does not prompt for pseudo-errors, so be sure to treat each warning carefully. However, this is closely related to your test, and it is important to ensure that the concurrent code is fully executed so that the race detector can play its part.

What are you waiting for? Run "Go test-race" on your code now!