Go Range Internal Implementation

This is a creation in Article, where the information may have evolved or changed.

Original: Go Range Loop Internals

The loop in Go is range very handy, but I always think it's a bit cryptic in different situations. Now it seems that I am not alone:

#golang Pop quiz:does This program terminate?
Func Main () {
V: = []int{1, 2, 3}
For I: = Range v {
v = append (V, i)
}
}

-dαve Cheney (@davecheney) January

Today's #golang gotcha:the Two-value range over an array does a copy. Avoid by ranging over the pointer instead.https://t.co/sbk667osva

-damian Gryski (@dgryski) January 3,

After personally testing the example of two great gods, I should now accept and remember these facts, but memorizing things is always easy to forget. Figuring out what the underlying principle is, is more conducive to deeper understanding and memory, let's start.

First step: Read the damn manual (read the fxxking Manual)

Read about range the documentation first. The syntax-related instructions can be found in the Go specification document "For statements with RANGE clause" section for...range . I will not copy the entire document, but will summarize some of the parts that are useful to us.

First, remind yourself of what we want to focus on:

for i := range a {    fmt.Println(i)}

Range variable

As we all know, the loop variable to the left of range (in the example above i ) can be assigned in the following ways:

• Equal Value Direct Assignment ( = )
• Short variable declaration Assignment ( := )

Of course, you can write nothing to completely ignore the values that the iteration iterates over.

If you use the short variable declaration ( := ), Go will reuse the declared variable in each iteration of the loop (only valid within the scope of the loop)

Range expression

rangeThe result of the expression on the right (in the example above a ) can be the following data types:

• Array
• Pointer to array
• Slice
• String
• Map
• Can receive transmissions channel , such as:chan int or chan<- int

The range expression is evaluated once before starting the loop . However, there is one exception:

<div class= "Tip" >
If you do a pointer to an array or an array range and use only the array index: You len(a) are only evaluatedat this time.
</div>

Just evaluating len(a) means that the expression a may be evaluated at compile time and then replaced by the compiler with a constant. Explanation of the specifications in the documentation len :

If the variable s is an array, pointer to an array, and there are no channel or function calls in the expression that can receive input (very much), these cases are s not evaluated, len(s) and cap(s) are constants; In addition and cap calls are not constants and s will be evaluated.

So what's the evaluated point here ? Unfortunately, no instructions were found in the documentation. Of course, I guess it's the exact execution of an expression until it can no longer be disassembled. In any case, the most important thing is that the range expression is executed completely before the beginning of the iteration. So how do you get an expression to be executed only once? Put the execution result in a variable! range does the processing of expressions do the same?

It is interesting to note that some of the maps additions or deletions to (not mentioned) are mentioned in the specification document slices .

If map the element is removed when it has not been traversed, the element will no longer appear in subsequent iterations. If map the element is added during the iteration, the element may or may not be skipped in subsequent iterations.

Just a moment, we'll come back and talk maps .

Step two: Data types supported by range

If we assume that the range expression is copied to a variable before the loop starts, what do we need to focus on? The answer is the data type of the result of the expression, so let's take a closer look at the range supported data types.

Before we start, remember: in Go, whatever assignment we make will be duplicated. If a pointer is assigned, we copy a copy of the pointer. If a struct is assigned, we copy a struct copy. The same is true for the transfer of parameters to a function. All right, let's go:

type	Who's Grammar candy?
Array	Array
String	A struct: An array with a variable len and a pointer pointing to the back
Slice	A struct: Having a variable len , a variable, cap and a pointer to an array behind it
Map	Pointer to a struct body
Channel	Pointer to a struct body

Some references are listed at the end of this article to learn more about the internal structure of these data types.

So what are these luǎn for? The following examples highlight some of the different points:

// 复制整个数组var a [10]intacopy := a // 只复制了 slice 的结构体，并没有复制成员指针指向的数组s := make([]int, 10)scopy := s// 只复制了 map 的指针m := make(map[string]int)mcopy := m

So, if you want to assign an array expression to a variable before the range loop starts (guaranteeing that the expression is only evaluate once), the entire array is copied.

Step three: Go compiler source code

Lazy as I direct Google a bit Go to the compiler source code. The first thing to discover is the GCC version of the compiler. The part that we care about with the range appears statements.cc , just like in the note:

// Arrange to do a loop appropriate for the type.  We will produce//   for INIT ; COND ; POST {//           ITER_INIT//           INDEX = INDEX_TEMP//           VALUE = VALUE_TEMP // If there is a value//           original statements//   }

Now it's finally a little bit. rangeloops in the internal implementation of the actual is the C-style cycle of grammatical sugar, unexpected and reasonable. The compiler makes range a special " Syntax sugar restore " for each type of support. Like what

Array:

// The loop we generate://   len_temp := len(range)//   range_temp := range//   for index_temp = 0; index_temp < len_temp; index_temp++ {//           value_temp = range_temp[index_temp]//           index = index_temp//           value = value_temp//           original body//   }

Slice:

//   for_temp := range//   len_temp := len(for_temp)//   for index_temp = 0; index_temp < len_temp; index_temp++ {//           value_temp = for_temp[index_temp]//           index = index_temp//           value = value_temp//           original body//   }

What they have in common is:

• All types are range essentially C-style loops
• Values that are traversed are assigned to a temporary variable

This is the case in Gofrontend, as far as I know, most people use the GC compiler that comes with the Go release, and they seem to have exactly the same behavior on this point.

All we know is that

1. Loop variables are assigned and reused in each iteration.
2. You can remove elements from a map or add elements to a map during the iteration. The added elements are not necessarily traversed in subsequent iterations.
3. Having made this clear, let's go back to the example listed in the opening.

Dave's Tweets

#golang Pop quiz:does This program terminate?
Func Main () {
V: = []int{1, 2, 3}
For I: = Range v {
v = append (V, i)
}
}

-dαve Cheney (@davecheney) January

This code terminates because it can actually be roughly translated into something like this:

for_temp := vlen_temp := len(for_temp)for index_temp = 0; index_temp < len_temp; index_temp++ {        value_temp = for_temp[index_temp]        index = index_temp        value = value_temp        v = append(v, index)}

We know that a slice is actually a syntactic sugar of a struct that has a pointer member to an array. A copy of the struct is then assigned to the structure before the loop starts, and the for_temp subsequent loop is actually iterating over the pair for_temp . Any change to the original variable v itself, rather than the array to which it is pointing, has no relation to the resulting copy for_temp . But the array behind it is still shared with pointers v for_temp , so v[i] = 1 such statements can still work.

Damian's Tweets

Today's #golang gotcha:the Two-value range over an array does a copy. Avoid by ranging over the pointer instead.https://t.co/sbk667osva

-damian Gryski (@dgryski) January 3,

Similar to the example above, the array is assigned a temporary variable before the loop starts, and when the array range is looped, the temporary variable holds a copy of the entire array , and the operation on the original array is not reflected on the copy. When the array pointer range is looping, the temporary variable holds a copy of the pointer , and the same memory space is manipulated.

Attached: Maps

In the specification document we read:

• range maps It is safe to add or remove elements in a loop.
• If an element is added to the loop maps , the element does not necessarily appear in subsequent iterations.

for the 1th , we know that maps it is actually a pointer to a struct. Before the loop starts, only the pointer is copied instead of the internal data structure, so adding or removing elements in the loop is consistent with the original variable and is reasonable.

Why is it not possible to traverse the currently added element in subsequent iterations ? If you know how a hash table works (the map is essentially a hash table), you will understand that the elements inside the hash table are not stored in a particular order. The last added element has the potential to be hashed into the first index bit in the inner array, and we really have no way of predicting whether the currently added element will appear in the subsequent iterations, after all, it is likely that the first index bit has been traversed when the element is added. Therefore, whether the currently added element can be traversed in subsequent iterations, or the mood of the compiler:D

Reference

• The Go Programming Language specification
• Go Slices:usage and Internals
• GO Data Structures
• Inside the Map Implementation:slides | Video
• Understanding Nil:slides | Video
• String source code
• Slice source code
• Map Source Code
• Channel Source code

Original: Go Range internal implementation