How the Go language implements race Dectect

This is a creation in Article, where the information may have evolved or changed.

How the Go language implements race Dectect

2016-11-20

How the Go language implements race Dectect

1.

Before writing if detection race, first understand first question, what is race?

When multiple goroutine simultaneously perform read-and-write collisions on the same variable, the result is not deterministic, which is race. For example, goroutine1 reading A,goroutine2 in writing a, if not sure goroutine1 read the result is Goroutine2 write before or after the value, is race.

var x intgo func() {    v := x}()x = 5

Is the value of the above code v exactly 0, or 5? I don't know, this code exists race. This is a more verbal description, rigorous formalization of the description, you need to talk about the memory model of go.

Go's memory model describes a condition that reads a variable in a groutine to detect write operations on the variable in another goroutine.

It doesn't unfold here, I can read some of the things I wrote before (Tl;dr ... Because the old and not updated, the inside of the example is still wrong now ... But this article is good for understanding the happens before concept.

Suppose A and B represent two operations performed by a multithreaded program. If a Happens-before B, then the effect of a operation on memory will be visible to the thread executing B (and before B is executed).

With happens before so formal description, whether there is race, equivalent to the same piece of memory access, there is no way to judge happens before conflict operation. That is to say:

For the previous piece of code, v := x and x = 5 two operations access to the same piece of memory X, and no guarantee v := x is happens before x = 5 , so this code has race.

Then the problem of "realizing race Dectect" is transformed into a "detection problem of happens before event".

2.

How to detect the happens before event?

We can put "which thread ID, at what time, which block of memory to access, is read or write", as long as all memory access events are recorded, and then traverse, verify the sequence of these operations. Once found, for example, read and write two operation Records, can not meet the read happens before write, is detected race.

But to log all of the memory access operations, it looks like the price is a bit scary. In fact, just record the variables that may be accessed concurrently, not all variables, and the next g is a local variable, it doesn't need to be recorded.

func f() {    g := 3}

But it still seems to be a big price? Do. Okay, it's going to be 10 times or 100 times times slower. I'm not sure, but the line code is not going to run race. Now that go has been done, you can certainly do it.

Two parts are required, and the -race compile option in Go will be processed accordingly. The compilation section needs to insert instructions to log events where memory accesses are involved, while the runtime detects happens before between events.

The whole idea is this, and then the details are detailed.

3.

A memory access event can be recorded in 8 bytes: 16-bit thread id,42 bit timestamp, 5-bit memory location, 1-bit token is read or write.

Thread IDs are not explained, and read-write tags are not explained. The timestamp is the logical clock, not the actual time taken at a time.

How do you log memory locations with only 5 bits? There is a bit of finesse here, and the same technique is used for go memory management. For an area of memory that is actually used, mapping another piece of "Shadow" memory area, mapping out the real "shadow".

For example, there is an array of a[1000], and its "shadow" is b[1000]. A[i] What happened inside, only in the record in B[i] in the line. Note that the two sizes do not need to be the same, such as

int  A[1000];   // 真实使用的数组char B[1000];   // 用于记录发生在A数组里面操作，如果只记读/写1位足已，记其它也不一定用到8位

Similarly, for the area of memory actually used is [0X7FFFFFFFFFFF 0x7f0000000000], its "shadow" area can be [0x1fffffffffff 0x180000000000],5 bit can represent 64 units, If the actual memory used is used by 8-byte alignment, it is sufficient to represent a group.

Seems to be a little confused, so explain it: 3 bits can represent the state of 8 units, right? 2 of 3 equals 8.

A[8个8字节的单元] => B[3位]

Whether a read or write operation has occurred in a, in B with a bit of 0 or 1 record. Note that a large number of events can be logged with only a small amount of memory!

Back to the record format of the event, a record occupies 8 bytes with 5 bits of memory location. The 5-bit is capable of recording 64 8-byte, that is, the space overhead of the race dectect is 1/8 of the memory used (not actually, because of the event on the same memory, to record a group).

For example, we recorded the first event, thread T1, E1 timestamp, accessing the memory area [0 2], and performing a write operation:

(T1,E1,0:2,W)

The second event, thread T2, E2 timestamp, read memory area [4 8]:

(T2,E2,4:8,R)

There is no conflict because the location has no intersection.

The third event, thread T3, E3 timestamp, read memory area [0 4]:

(T3,E3,0:4,R)

This area is intersected with the area of the first event, so if E1 cannot meet happens before E3, then a conflict is detected.

Finish.

Reference: Https://github.com/google/sanitizers/wiki/ThreadSanitizerAlgorithm

Thinking is a very interesting thing. The beginning is the curiosity, the end is the wisdom.

Thinking requires a step-by-step abstraction of a specific problem into an implementation that can be described in code.

Can not help but read the answer, so the order of writing is a step by step backward. In fact, when you do not know how to solve a thing, I think, is very able to exercise the ability of abstract problems, but also very interesting. Readers who have finished reading this article will miss a chance.

So, to stay a study questions, deadlock detection and how to do it? Can leave a message, or a resume for everyone to exchange (a joke, do not advertise).