Golang slicing and function parameter "traps"

This is a creation in Article, where the information may have evolved or changed.

Linear structure is one of the most commonly used data structures in computers. Whether it is an array (arrary) or a linked list (list), it is indispensable in programming. Golang also has arrays, unlike other languages, Golang also provides slices (slice). Slices have better flexibility than arrays and have some dynamic characteristics. However, slicing is not like a list of dynamic languages (Python list). Do not understand the basic implementation of the slice, write the program when it is easy to drop the "pit."

Slice parameters

Originally wrote a heap sort, uses the Golang slice to do the heap, but discovered in the pop data time, the slice does not change. Then the parameters of the Golang function slice are raised, are the values passed or quoted? We know that slice is a reference type compared to array. So intuitively it tells us that if the function modifies the slice of the parameter, the outer slice variable will change.

func main() {    slice := []int{0, 1, 2, 3}    fmt.Printf("slice: %v slice addr %p \n", slice, &slice)    ret := changeSlice(slice)    fmt.Printf("slice: %v ret: %v slice addr %p \n", slice, &slice, ret)}func changeSlice(s []int) []int {    s[1] = 111    return s}

The result is the same as the assumption:

slice: [0 1 2 3], slice addr: 0xc4200660c0 slice: [0 111 2 3], ret: [0 111 2 3], slice addr: 0xc4200660c0

changeSliceThe function modifies the slice, and the variable slice is modified as well. However, if it is easy to conclude that the slice parameters are passed by reference, then the following phenomenon requires a statement:

func changeSlice(s []int) []int {    fmt.Printf("func: %p \n", &s)    s[1] = 111    return s}

We are typing the address of the parameter S in the function, and we can see that this address is not the same as the slice in the main function. To understand this, we need to understand the basic implementation of slice in Golang.

Slice BASIC Implementation

The slice in Golang is a compound structure that looks like an array but not an array. The slice, as its name implies, is a fragment of the array cut down. The slice structure roughly stores three parts, the first part is a pointer to the underlying array ptr , followed by the size of the slice len and the capacity of the slice cap :

      +--------+      |        |      |  ptr   |+------------+-------+-----------+      |        |                     |           |      +--------+                     |           |      |        |                     |           |      |        |                     |           |      |  len 5 |                     |           |      |        |                     |           |      +--------+                     v           v      |        |             +-----+-----+-----+-----+----+      |        |             |     |     |     |     |    |      |  cap 5 |     [5]int  |  0  |  1  |  2  |  3  | 4  |      |        |             +-----+-----+-----+-----+----+      +--------+       slice := arr[1:4]             arr := [5]int{0,1,2,3,4}

An array arr is a structure that contains five int types, and its slice slice only takes 1 to 3 of these numbers. We can also regenerate into a slice slice2 := arr[2:5] , taking the contiguous block behind the array. Together, they use ARR as the underlying structure, and can see the 3rd, 4 elements that share the numbers. Modifying any of these can change the value of two slices.

func main() {    arr := [5]int{0, 1, 2, 3, 4}    fmt.Println(arr)    slice := arr[1:4]    slice2 := arr[2:5]    fmt.Printf("arr %v, slice1 %v, slice2 %v, %p %p %p\n", arr, slice, slice2, &arr, &slice, &slice2)    fmt.Printf("arr[2]%p slice[1] %p slice2[0]%p\n", &arr[2], &slice[1], &slice2[0])    arr[2] = 2222    fmt.Printf("arr %v, slice1 %v, slice2 %v\n", arr, slice, slice2)    slice[1] = 1111    fmt.Printf("arr %v, slice1 %v, slice2 %v\n", arr, slice, slice2)}

The output value is:

[0 1 2 3 4]arr [0 1 2 3 4], slice1 [1 2 3], slice2 [2 3 4], 0xc42006e0c0 0xc4200660c0 0xc4200660e0arr[2]0xc42006e0d0 slice[1] 0xc42006e0d0 slice2[0]0xc42006e0d0arr [0 1 2222 3 4], slice1 [1 2222 3], slice2 [2222 3 4]arr [0 1 1111 3 4], slice1 [1 1111 3], slice2 [1111 3 4]

Thus, the array of slices, just cut a piece of data from the array, different slices, in fact, is to share these underlying data. However, the slices themselves are different objects and their memory addresses are different.

Slicing a piece down from an array to form a slice well understood, sometimes we create slices with the Make function, and in fact Golang creates an anonymous array at the bottom. If you re-cut from the new slice, the newly created two slices share the underlying anonymous array.

func main() {    slice := make([]int, 5)    for i:=0; i<len(slice);i++{        slice[i] = i    }    fmt.Printf("slice %v \n", slice)    slice2 := slice[1:4]    fmt.Printf("slice %v, slice2 %v \n", slice, slice2)    slice[1] = 1111    fmt.Printf("slice %v, slice2 %v \n", slice, slice2)}

The output is as follows:

slice [0 1 2 3 4] slice [0 1 2 3 4], slice2 [1 2 3] slice [0 1111 2 3 4], slice2 [1111 2 3]

Replication of Slice

Since the creation of slice depends on the array, sometimes the newly generated slice is modified, but does not want to modify the original slice or array. At this point, you need to copy the original slices.

func main() {    slice := []int{0, 1, 2, 3, 4}    slice2 := slice[1:4]    slice3 := make([]int, len(slice2))    for i, e := range slice2 {        slice3[i] = e    }    fmt.Printf("slice %v, slice3 %v \n", slice, slice3)    slice[1] = 1111    fmt.Printf("slice %v, slice3 %v \n", slice, slice3)}

Output:

slice [0 1 2 3 4], slice3 [1 2 3] slice [0 1111 2 3 4], slice3 [1 2 3]

Thus, the newly created slice3 will not change slice3 because of changes in slice and slice2. Replication is useful, so Golang implements a built-in function copy , copy has two parameters, the first parameter is the copied object, and the second is the array slice object before copying.

func main() {    slice := []int{0, 1, 2, 3, 4}    slice2 := slice[1:4]    slice4 := make([]int, len(slice2))    copy(slice4, slice2)    fmt.Printf("slice %v, slice4 %v \n", slice, slice4)    slice[1] = 1111    fmt.Printf("slice %v, slice4 %v \n", slice, slice4)}

Slice4 is generated from the Slice2 copy, and the anonymous array slice and Slice4 is not the same. So modify them without affecting each other.

Slice Append

Append Introduction

Creating a copy slice is a common operation, with an append element or an append array, which is also a common feature. Golang provides a append function to append elements to a slice. Append the first parameter is the original slice, followed by some mutable arguments, for the element or elements that will be appended.

func main() {    slice := make([]int, 1, 2)    slice[0] = 111    fmt.Printf("slice %v, slice addr %p, len %d, cap %d \n", slice, &slice, len(slice), cap(slice))    slice = append(slice, 222)    fmt.Printf("slice %v, slice addr %p, len %d, cap %d \n", slice, &slice, len(slice), cap(slice))    slice = append(slice, 333)    fmt.Printf("slice %v, slice addr %p, len %d, cap %d \n", slice, &slice, len(slice), cap(slice))}

The output is:

slice [111], slice addr 0xc4200660c0, len 1, cap 2 slice [111 222], slice addr 0xc4200660c0, len 2, cap 2 slice [111 222 333], slice addr 0xc4200660c0, len 3, cap 4

Slice capacity

Regardless of the array or slice, there is a length limit. When appending slices, if the element is within the capacity of the slice, an element is appended directly to the tail. If the maximum capacity is exceeded, then appending the element requires copying and scaling for the underlying array.

Here is the concept of slicing capacity, which cuts the data from the array, the slice's capacity should be the last data of the slice, and the size of the remaining elements, plus the size of the existing tiles.

Array [0, 1, 2, 3, 4], the array has 5 elements. If the slice s = [1, 2, 3], then the index of the 3 array is 3, that is, the array remains the size of the last element, plus s already has 3 elements, so the last s capacity is 1 + 3 = 4. If the slice is
S1 = [4],4 index is the largest in the array, the array is free of 0 elements, then the capacity of S1 is 0 + 1 = 1. Specifically, the following table:

slices	Slice literal	Array left space	length	capacity
S[1:3]	[1 2]	2	2	4
S[1:1]	[]	4	0	4
S[4:4]	[]	1	0	1
S[4:5]	[4]	0	1	1

Although the second and third slices above are the same length, their capacity is different. Capacity is related to the strategy of final append.

Append Simple Implementation

We already know that slices rely on the underlying array structure, even if they are created directly, an anonymous array is generated. When using append, it essentially operates on an array of underlying dependencies. If the slice's capacity is greater than the length, appending the element to the slice actually modifies the element that follows the slice element in the underlying number. If the capacity is full, it is not possible to modify the original array, but to create a new array, of course Golang is implemented by creating a new slice, because the new slice must also have a new array, and the length of the array is twice times the original, using the simple implementation of the dynamic programming algorithm.

func main() {    arr := [3]int{0, 1, 2}    slice := arr[1:2]    fmt.Printf("arr %v len %d, slice %v  len %d, cap %d, \n", arr, len(arr), slice, len(slice), cap(slice))    slice[0] = 333    fmt.Printf("arr %v len %d, slice %v  len %d, cap %d, \n", arr, len(arr), slice, len(slice), cap(slice))    slice = append(slice, 4444)    fmt.Printf("arr %v len %d, slice %v  len %d, cap %d, \n", arr, len(arr), slice, len(slice), cap(slice))    slice = append(slice, 5555)    fmt.Printf("arr %v len %d, slice %v  len %d, cap %d, \n", arr, len(arr), slice, len(slice), cap(slice))    slice[0] = 333    fmt.Printf("arr %v len %d, slice %v  len %d, cap %d, \n", arr, len(arr), slice, len(slice), cap(slice))}

Output:

arr [0 1 2] len 3, slice [1]  len 1, cap 2, arr [0 333 2] len 3, slice [333]  len 1, cap 2, arr [0 333 444] len 3, slice [333 444]  len 2, cap 2, arr [0 333 444] len 3, slice [333 444 555]  len 3, cap 4, arr [0 333 444] len 3, slice [333 444 555]  len 3, cap 4,

Append less than the capacity

Re-export, let's draw a diagram of this dynamic process:

       +----+----+----+                           +----+----+----+                             +----+----+----+       |    |    |                           |    |    |    |                             |    |    |    | | Arr |  0 |  1 |                     2 | Arr | 0 |333 |                       2 | Arr |       0 |333 |444 |                                          +----+----+----+                           +----+----+----+                             +----+----+----+               ^                                          ^                                            ^    ^               |                                            |    |               |                                          |                                            |    |               |                 |              SLIC0] = 333 |               Slice = append (slice, 444) +----+ |                +-----------------> |          +----------------> |     |                                            |            |     +--+--+----+----+                          +--+--+----+----+                            +--+--+----+----+            |    |    |                          |     |    |    |                            |     |    |    |            | | P | 1 |                          2 | | P | 1 |                            2 | | P | 2 |            2 |  +-----+----+----+ +-----+----+----+ +-----+----+----+ Slice : =arr[1:2] Slice: =arr[1:2] Slice: =arr[1:2]

Arr is an array of three elements, slice an element from arr, since the last element of the slice is the index of the 1 array 1 , the maximum length of the distance array is 1, so the slice capacity is 2. When the first element of the slice is modified, the second element of ARR is modified accordingly because the slice is an arr array. When appending an element to slice using the Append method, because the slice capacity is not yet full, it is equivalent to extending the contents of the slice pointer to the array, which can be understood as a re-slicing of the contents of an array appended to the slice, while modifying the contents of the array.

Append of excess capacity

If you then append an element, the array must be out of bounds. At this point the principle of append is roughly as follows:

1. Create a new temporary tile the length of the t,t is the same as the length of the slice slice, but the capacity of T is twice times the size of the slice slice, a dynamically planned way. When you create a new slice, the bottom layer also creates an anonymous array with the same length as the tile size.
2. Copy the elements inside the s into the T, that is, fill in the anonymous array. Then assign T to slice, and now slice to the underlying anonymous array.
3. into a append method that is less than the capacity.

       +----+----+----+                                +----+----+----+----+----+----+       |    |    |                                |    |    |    |    |    |    | | Arr |                                0 |333 |444 | | 333|    444|    |    |    |       |                                    +----+----+----+                                +----+----+----+----+----+----+               ^    ^    ^     ^               |                                    |     |               |    |                                    |               +-----+ +----+ +---------------> |                                         |               |                                         |     +            +--+--+----+----+                          +-----+-----+-----+            |    |    |                          |     |     |     |            | | P | 2 |                          2 | |  P |  2 |            6 | +-----+----+----+                          +-----+-----+-----+ slice: =arr[1:2] t: = make ([]int, len=2, cap=6)                                                                  +                                                                  |                                                                  |                                                                  |                                                                  |                                                       V +----+----+----+----+----+----+    |    |    |    |    |    |                                                       | | 333|    444|555 |    |    |                                                       |                                                         +----+----+----+----+----+----+                                                         ^         ^         |                         |                                +----+----+                                                         |                                                         |                                                       +                                                       +-----+-----+-----+     |     |     |                                                       | |  P |  3 |                                                       6 | +-----+-----+-----+ slice = t

The above diagram depicts the operating principle of append when it is larger than the capacity. The newly generated slice has no relation to its dependent array and the original array, so the old array is not related to the new slice element being modified. As for the temporary slice T, it will be recycled by Golang GC. Of course arr or its derived slices are not applied and will be recycled by the GC.

The relationship between slice and array is very close, through the rational construction of both, can realize dynamic and flexible linear structure, but also can provide the efficient performance of access elements. Of course, this structure is not perfect, sharing the underlying array, when the partial modification operation, may bring side effects, and if a large array, it is afraid that only one element is applied to the slice, then the remaining array will not be garbage collection, which often brings additional problems.

Slicing as a function parameter

Change slices directly

Back to the beginning of the question, when the function's parameters are slices, is it a pass or a reference? From the Changeslice function, the address of the argument s, you can see that is certainly not a reference, after all, the reference is an address. However, the value of S is changed within the Changeslice function, and the value of the original variable slice, which looks like a reference phenomenon, is actually the implementation of the slice sharing underlying array we discussed earlier.

That is, when a slice is passed, it is the value of the array, which is equivalent to cutting it again from the original slice. The original slice slice is the same as the underlying array for the parameter S slice. As a result, the array is modified by modifying the slices within the function.

                                                +-----+----+-----+                                                |     |    |     |                 +-----------------------------+| p   |  3 |  3  |                 |          +                   +-----+----+-----+                 |          |                 |          |                     s                 |          |                 |          |                 v          v               +----+----+-----+               |    |    |     |       arr     | 0  |  1 |  2  |               +----+----+-----+                 ^           ^                 |           |                 |           |                 +-----------+                 |                 |               +-+--+----+-----+               |    |    |     |               |  p |  3 |  3  |               +----+----+-----+                  slice

For example, the following code:

    slice := make([]int, 2, 3)    for i := 0; i < len(slice); i++ {        slice[i] = i    }    fmt.Printf("slice %v %p \n", slice, &slice)    ret := changeSlice(slice)    fmt.Printf("slice %v %p, ret %v \n", slice, &slice, ret)    ret[1] = 1111    fmt.Printf("slice %v %p, ret %v \n", slice, &slice, ret)}func changeSlice(s []int) []int {    fmt.Printf("func s %v %p \n", s, &s)    s = append(s, 3)    return s}

Output:

slice [0 1] 0xc42000a1e0 func s [0 1] 0xc42000a260 slice [0 1] 0xc42000a1e0, ret [0 1 3] slice [0 1111] 0xc42000a1e0, ret [0 1111 3]

As you can see from the output, when slice is passed to the function, a new slice s is created. A append element is given to s in the function, because the capacity of S is sufficient at this time, and no new underlying array is generated. When modifying the returned RET, RET also shared the underlying array, so modify the original RET, the corresponding also saw the slice change.

Append operation

If the append operation exceeds the capacity of the original slice within the function, there will be a new procedure for creating the underlying array, and then modifying the function to return the slice should no longer affect the original slice. For example, the following code:

 func main() {    slice := make([]int, 2, 2)    for i := 0; i < len(slice); i++ {        slice[i] = i    }    fmt.Printf("slice %v %p \n", slice, &slice)    ret := changeSlice(slice)    fmt.Printf("slice %v %p, ret %v \n", slice, &slice, ret)    ret[1] = -1111    fmt.Printf("slice %v %p, ret %v \n", slice, &slice, ret)}func changeSlice(s []int) []int {    fmt.Printf("func s %v %p \n", s, &s)    s[0] = -1    s = append(s, 3)    s[1] =  1111    return s}

Output:

slice [0 1] 0xc42000a1a0 func s [0 1] 0xc42000a200 slice [-1 1] 0xc42000a1a0, ret [-1 1111 3] slice [-1 1] 0xc42000a1a0, ret [-1 -1111 3]

From the output we can clearly see our conjecture. That is, the function first changes the value of the first element of S, since slice and s both share the underlying array, so whether the original slice slice or RET, The first element is-1. Then after the append operation, because the capacity of S is exceeded, a new underlying array is created, although the s variable is unchanged, but his underlying array is changed, and the first element of S is modified, without affecting the original slice slice. That is slice[1] or 1, while ret[1] is-1. Finally, modify the ret[1] to 1111, and it will not affect the original slice slice.

From the above analysis, we can generally conclude that slice or array as function parameters, the essence is to pass the value rather than to pass the reference. The process of passing a value copies a new slice, which also points to the underlying array of the original variable. (Personal feeling is that a slice may be more accurate than the value of a pass). The function is based on the case of a shared slice underlying array, whether directly modifying the slice or append creating a new slice. That is, whether the outermost original slice changes, depending on the operation within the function and the size of the slice itself.

Pass-by-reference method

The process of array and slice passing as parameters is essentially the same as passing them slices. Sometimes we need to deal with the form of passing references. Golang provides pointers that are easy to implement similar functions.

func main() {    slice := []int{0, 1}    fmt.Printf("slice %v %p \n", slice, &slice)    changeSlice(&slice)    fmt.Printf("slice %v %p \n", slice, &slice)    slice[1] = -1111    fmt.Printf("slice %v %p \n", slice, &slice)}func changeSlice(s *[]int) {    fmt.Printf("func s %v %p \n", *s, s)    (*s)[0] = -1    *s = append(*s, 3)    (*s)[1] =  1111}

The output is as follows:

slice [0 1] 0xc42000a1e0 func s [0 1] 0xc42000a1e0 slice [-1 1111 3] 0xc42000a1e0 slice [-1 -1111 3] 0xc42000a1e0

From the output can be seen, passed to the function is a pointer to slice, the function of the operation of the S is essentially the operation of the slice. It can also be seen from the S address that is typed from within the function, with only one slice from the beginning to the end. Although a temporary slice or array appears during the append process.

Summarize

Golang provides a sequence structure of array and slice two. Where array is a value type. The slice is a composite type. The slice is based on array implementations. The first content of slice is a pointer to an array, followed by its length and capacity. You can use the slice of the array to 切 create a slice with make, and Golang generates an anonymous array.

Because slice relies on its underlying array, modifying slice essentially modifies the array, and the array has a size limit, and when it exceeds the capacity of slice, that is, when the array is out of bounds, a new array block needs to be created in a dynamically planned way. Copy the original data to the new array, which is the new underlying dependency of slice.

An array or a slice, passed in a function that is not a reference, is another value type that is passed in through the original variable 切片 . The operation within the function is the modification of the slice. Of course, if you want to modify the original variable, you can specify the type of the parameter as the pointer type. Passing is the memory address of the slice. The action inside the function is to find the variable itself based on the memory address.

Reference: Go slices: usage and nature