Go Unsafe Package

This is a creation in Article, where the information may have evolved or changed.

Go Unsafe Package

Unsafe Package Overview

Until now (Go1.7), unsafe contains the following resources:

Three functions:

// unsafe.Sizeof函数返回操作数在内存中的字节大小,参数可以是任意类型的表达式,但是它并不会对表达式进行求值.// 一个Sizeof函数调用是一个对应uintptr类型的常量表达式,// 因此返回的结果可以用作数组类型的长度大小，或者用作计算其他的常量.func Sizeof(x ArbitraryType) uintptr//函数的参数必须是一个字段 x.f, 然后返回 f 字段相对于 x 起始地址的偏移量, 包括可能的空洞.func Offsetof(x ArbitraryType) uintptr//unsafe.Alignof 函数返回对应参数的类型需要对齐的倍数.func Alignof(x ArbitraryType) uintptr

A memory hole is an unused memory space that is automatically added by the compiler to ensure that the address of each subsequent field or element is reasonably aligned with respect to the starting address of the structure or array (the memory hole may have some random data that can affect the handling of the memory directly with the unsafe packet).

and a type:

type Pointer *ArbitraryType

Here, Arbitrarytype is not a real type. The official export of this type is only for the sake of perfect documentation, and is not useful in other libraries or any projects unless the programmer intentionally uses it.

Unsafe. Sizeof, Alignof and Offsetof

When the computer loads and saves data, it is more efficient if the memory address is properly aligned. For example, the variable address of a 2-byte int16 type should be even, the address of a 4-byte rune type variable should be a multiple of 4, and the address of a 8-byte float64, UInt64, or 64-bit pointer-type variable should be 8-byte aligned. However, it is not necessary for a large number of address alignments, even for large data types such as complex128, which are only 8-byte aligned.

because of the address alignment, the size of an aggregate type (struct or array) is at least the sum of all fields or element sizes, or greater because of the possibility of a memory hole. A memory hole is an unused memory space that is automatically added by the compiler to ensure that the address of each subsequent field or element is reasonably aligned with respect to the starting address of the structure or array (there may be some random data in the memory hole, may have an impact on handling memory directly with unsafe packets).

A struct variable x and its typical memory on a 64-bit machine. The gray area is empty.

var x struct {a boolb int16c []int}

The calculation results of the three fields of the struct variable called unsafe packet correlation function are as follows.

package mainimport ("fmt""unsafe")func main() {var x struct {a boolb int16c []int}//通常情况下布尔和数字类型需要对齐到它们本身的大小(最多8个字节),其它的类型对齐到机器字大小.(64位的机器字大小为64位,8字节)fmt.Printf("%-30s%-30s%-30s%-50s\n","Row", "Sizeof", "Alignof(对齐倍数)", "Offsetof(偏移量)")fmt.Printf("%-30s%-30d%-30d%-50s\n","x", unsafe.Sizeof(x), unsafe.Alignof(x), "")fmt.Printf("%-30s%-30d%-30d%-50d\n","x.a", unsafe.Sizeof(x.a), unsafe.Alignof(x.a), unsafe.Offsetof(x.a))fmt.Printf("%-30s%-30d%-30d%-50d\n","x.b", unsafe.Sizeof(x.b), unsafe.Alignof(x.b), unsafe.Offsetof(x.b))fmt.Printf("%-30s%-30d%-30d%-50d\n","x.c", unsafe.Sizeof(x.c), unsafe.Alignof(x.c), unsafe.Offsetof(x.c))}

Run the results,

Row                           Sizeof                        Alignof(对齐倍数)                 Offsetof(偏移量)                                     x                             32                            8                                                                               x.a                           1                             1                             0                                                 x.b                           2                             2                             2                                                 x.c                           24                            8                             8                            

Unsafe. Pointer

Most pointer types are written as *t, which means "a pointer to a variable of type T". Unsafe. Pointer is a specially defined pointer type (like a pointer to a void* type in C) that can contain the address of any type of variable. Of course, we are not allowed to get unsafe directly through *p. The pointer pointer points to the value of the real variable, because we do not know the exact type of the variable. As with normal pointers, unsafe. The pointer pointer is also comparable and supports the comparison with the nil constant to determine if it is a null pointer.

A normal *t type pointer can be converted to a Unsafe.pointer type pointer, and a unsafe.pointer type pointer can also be reversed to a normal pointer, and the normal pointer type does not need to be the same as the original *t type. By converting the *float64 type pointer to the *uint64 type pointer, we can view the bit pattern of a floating-point variable.

package mainimport (    "fmt"    "unsafe"    "reflect")func Float64bits(f float64) uint64 {    fmt.Println(reflect.TypeOf(unsafe.Pointer(&f)))  //unsafe.Pointer    fmt.Println(reflect.TypeOf((*uint64)(unsafe.Pointer(&f))))  //*uint64    return *(*uint64)(unsafe.Pointer(&f))}func main() {    fmt.Printf("%#016x\n", Float64bits(1.0)) // "0x3ff0000000000000"}

By moving to a new type pointer, we can update the bit pattern of the floating-point number. It is possible to manipulate floating-point numbers through bit mode, but the more important point is that the pointer conversion syntax allows us to write arbitrary values to memory without destroying the type system.

an unsafe. The pointer pointer can also be converted to the UIntPtr type and then saved to a pointer-type numeric variable (this is just a numeric value that is the same as the current pointer, not a pointer), and is then used to do the necessary pointer-numeric operations. (chapter III, UINTPTR is an unsigned integer that is sufficient to hold an address) This conversion is reversible, but turns uintptr into unsafe. The pointer pointer may break the type system because not all numbers are valid memory addresses.

Many will be unsafe. Pointer the pointer to a native number and then back to unsafe. The operation of the pointer type pointer is also unsafe. For instance, the following example needs to convert the address of the variable x with the B-field address offset to the *int16 type pointer, and then update the x.b with that pointer:

package mainimport ("fmt""unsafe")func main() {var x struct {a boolb int16c []int}// 和 pb := &x.b 等价pb := (*int16)(unsafe.Pointer(uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)))*pb = 42fmt.Println(x.b) // "42"}

The above wording, though cumbersome, is not a bad thing here, because these functions should be used with caution. Do not attempt to introduce a temporary variable of type uintptr because it can break the security of your code (this is an example of why unsafe packets are unsafe). The following snippet is wrong:

package mainimport ("fmt""unsafe")func main() {var x struct {a boolb int16c []int}// NOTE: subtly incorrect!tmp := uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)pb := (*int16)(unsafe.Pointer(tmp))*pb = 42fmt.Println(x.b) // "42"}

The cause of the error is subtle. Sometimes the garbage collector moves some variables to reduce memory fragmentation and so on. This type of garbage collector is called a mobile GC. When a variable is moved, all pointers to the old address that are saved must be updated to the new address after the variable is moved. From the garbage collector's point of view, an unsafe. Pointer is a pointer to a variable, so the pointer must also be updated when the variable is moved, but the temporary variable of the UIntPtr type is just a normal number, so its value should not be changed. The error code above is due to the introduction of a non-pointer temp variable, TMP, which prevents the garbage collector from correctly identifying this as a pointer to the variable x. When the second statement executes, the variable x may have been shifted, and the TEMP variable, TMP, is no longer the current &x.b address. The third assignment statement to the previously invalid address space will completely destroy the entire program!

===================

Unsafe. Pointer rules of Use,

(1) Any type of pointer can be converted to pointer

(2) pointer can be converted to any type of pointer

(3) UIntPtr can be converted to pointer

(4) pointer can be converted to UIntPtr

As an example:

package mainimport ("unsafe""fmt")func main() {var n int64 = 5var pn = &nvar pf = (*float64)(unsafe.Pointer(pn))fmt.Println(*pf) //2.5e-323*pf = 3.1415fmt.Println(n) //4614256447914709615}

The conversion in this example may be meaningless, but it is safe and legal.

UIntPtr and unsafe. Pointer the conversion of the other,

package mainimport ("unsafe""fmt")func main() {a := [4]int{0, 1, 2, 3}p := &a[1] // 内存地址p1 := unsafe.Pointer(p) p2 := uintptr(p1)p3 := unsafe.Pointer(p2)fmt.Println(p1) // 0xc420014208fmt.Println(p2) // 842350543368fmt.Println(p3) // 0xc420014208}

UIntPtr

About UIntPtr,

// uintptr is an integer type that is large enough to hold the bit pattern of// any pointer.type uintptr uintptr

The underlying implementation of UINTPTR is as follows, in $goroot/src/pkg/runtime/runtime.h,

#ifdef _64BITtypedef uint64          uintptr;typedef int64           intptr;typedef int64           intgo; // Go's inttypedef uint64          uintgo; // Go's uint#elsetypedef uint32          uintptr;typedef int32           intptr;typedef int32           intgo; // Go's inttypedef uint32          uintgo; // Go's uint#endif

UIntPtr and IntPtr are unsigned and signed pointer types, and ensure that 64-bit platforms are 8 bytes, 4 bytes on 32-bit platforms, and UINTPTR is primarily used for pointer operations in Golang.

Legal Case 1: converting between []t and []MYT]

In this example, we use int as T:

type MyInt int

In Golang, []int and []myint are two different types. Therefore, the value of []int cannot be converted to []myint, and vice versa. But in unsafe. With the help of pointer, conversion is possible:

package mainimport ("unsafe""fmt")func main() {type MyInt inta := []MyInt{0, 1, 2}// b := ([]int)(a) // error: cannot convert a (type []MyInt) to type []intb := *(*[]int)(unsafe.Pointer(&a))b[0] = 3fmt.Println("a =", a) // a = [3 1 2]fmt.Println("b =", b) // b = [3 1 2]a[2] = 9fmt.Println("a =", a) // a = [3 1 9]fmt.Println("b =", b) // b = [3 1 9]}

Legitimate use Case 2: Calling a pointer-related function in the sync/atomic package

Most of the parameters and result types of the following functions in the Sync/atomic package are unsafe. Pointer or *unsafe. Pointer:

// CompareAndSwapPointer executes the compare-and-swap operation for a unsafe.Pointer value.func CompareAndSwapPointer(addr *unsafe.Pointer, old, new unsafe.Pointer) (swapped bool)// LoadPointer atomically loads *addr.func LoadPointer(addr *unsafe.Pointer) (val unsafe.Pointer)// StorePointer atomically stores val into *addr.func StorePointer(addr *unsafe.Pointer, val unsafe.Pointer)// SwapPointer atomically stores new into *addr and returns the previous *addr value.func SwapPointer(addr *unsafe.Pointer, new unsafe.Pointer) (old unsafe.Pointer)

To use these features, you must import unsafe packages.

package mainimport ("unsafe""fmt""sync/atomic""time""sync""log""math/rand")var data *stringfunc Data() string {p := (*string)(atomic.LoadPointer((*unsafe.Pointer)(unsafe.Pointer(&data))))if p == nil {return ""} else {return *p}}// set data atomicallyfunc SetData(d string) {atomic.StorePointer((*unsafe.Pointer)(unsafe.Pointer(&data)), unsafe.Pointer(&d))}func main() {var wg sync.WaitGroupwg.Add(200)for range [100]struct{}{} {go func() {time.Sleep(time.Second * time.Duration(rand.Intn(1000)) / 1000)log.Println(Data())wg.Done()}()}for i := range [100]struct{}{} {go func(i int) {time.Sleep(time.Second * time.Duration(rand.Intn(1000)) / 1000)s := fmt.Sprint("#", i)log.Println("====", s)SetData(s)wg.Done()}(i)}wg.Wait()fmt.Println("final data = ", *data)}

Reproduced:

Https://shifei.me/gopl-zh/ch13/ch13-02.html

Http://www.open-open.com/lib/view/open1391347613192.html

=======end=======