Go Language Practical Notes (26) | Go Unsafe Package Memory layout

This is a creation in Article, where the information may have evolved or changed.

"Go language Combat" reading notes, not to be continued, welcome to sweep code attention flysnow_org to the public or website http://www.flysnow.org/, the first time to see follow-up notes. If you feel helpful, share it with your friends and thank you for your support.

Unsafe, as the name implies, is not safe, go definition of the package name is also the meaning, let us as far as possible do not use it, if you use it, see the name, you will think as far as possible do not use it, or use it more carefully.

Although this package is not safe, but it also has its advantage, that is to bypass the go memory security mechanism, directly to read and write memory, so sometimes because of the need for performance, will take some risk to use the package, the memory operation.

sizeof function

SizeofThe function can return a type of memory size that is only relevant to the type, regardless of the content size of the variable stored by the type, such as a bool type that occupies one byte, and int8 also occupies one byte.

12345678

func main () {fmt. Println (unsafe. Sizeof (true )) fmt. Println (unsafe. Sizeof (int8 (0 )) fmt. Println (unsafe. Sizeof (int16 (10 )) fmt. Println (unsafe. Sizeof (int32 (10000000 )) fmt. Println (unsafe. Sizeof (int64 (10000000000000 )) fmt. Println (unsafe. Sizeof (int (10000000000000000 ))}}

For integers, the number of bytes used means that this type stores the size of a range of numbers, such as int8 occupies a byte, which is 8bit, so it can store a size range of -128~~127, that is, −2^ (n-1) to 2^ (n-1) −1,n for bit, Int8 indicates that 8bit,int16 represents 16bit, and so on.

For the type of int of the peace station, this depends on whether the platform is 32-bit or 64-bit and will take the largest. For example, my own test, the above output, will find that the size of int and int64 is the same, because I am a 64-bit platform of the computer.

1	func Sizeof (x Arbitrarytype) UIntPtr

The above is Sizeof the function definition, which receives a ArbitraryType type of parameter that returns a uintptr value of type. Here's ArbitraryType not to care about, he is just a placeholder, in order to consider the document exported the type, but generally do not use it, we just need to know that it represents any type, that is, our function can receive any type of data.

123	//Arbitrarytype are here for the purposes of documentation and are not actually //Part of the unsafe package. It represents the type of an arbitrary Go expression. type int

Alignof function

AlignofReturns the alignment value of a type, which can also be called an alignment factor or an alignment multiplier. The alignment value is a value related to memory alignment, reasonable memory alignment can improve the performance of memory read and write, the knowledge about memory alignment can refer to the relevant documents, this does not expand the introduction.

123456789101112131415161718192021222324

func main() {varBBOOLvari8int8varI16Int16varI64Int64varF32float32varSstringvarMMap[string]stringvarP *Int32Fmt. Println (unsafe. Alignof (b)) fmt. Println (unsafe. Alignof (i8)) fmt. Println (unsafe. Alignof (I16)) fmt. Println (unsafe. Alignof (i64)) fmt. Println (unsafe. Alignof (F32)) fmt. Println (unsafe. Alignof (s)) fmt. Println (unsafe. Alignof (m)) fmt. Println (unsafe. Alignof (P))}

From the output of the above example, it can be seen that the alignment value is generally 2^n, the maximum will not exceed 8 (cause see the following memory alignment rules). Alignoffunction definition and Sizeof basically the same. The thing to note here is that everyone's computer may run differently, much the same.

1	func Alignof (x Arbitrarytype) UIntPtr

In addition, getting the alignment value can also use a function that reflects the package, which is said: unsafe.Alignof(x) equivalent to reflect.TypeOf(x).Align() .

Offsetof function

OffsetofThe function is only applicable to the memory location offset of the field in the struct struct relative to the struct body. The first field of the struct has an offset of 0.

1234567891011121314

func Main () {var u1 user1fmt. Println (unsafe. Offsetof (u1.b)) fmt. Println (unsafe. Offsetof (u1.i)) fmt. Println (unsafe. Offsetof (U1.J))}typestructbyteint32int64}

The offset of the field is the starting position of the field in the body layout of the struct structure (the memory location index starts at 0). Based on the offset of the field, we can locate the structure of the field, and then can read and write the structure of the field, even if they are private, the hacker feel there is no. The concept of offset is described in detail in the next summary.

In addition, it is unsafe.Offsetof(u1.i) equivalent toreflect.TypeOf(u1).Field(i).Offset

Interesting struct size

We define a struct, which has 3 fields, their types byte , int32 and, however, the int64 order of the three fields we can arrange arbitrarily, then according to the order of the different, a total of 6 combinations.

1234567891011121314151617181920212223242526272829303132333435

typeUser1struct{bbyteIInt32JInt64}typeUser2struct{bbyteJInt64IInt32}typeUser3struct{iInt32BbyteJInt64}typeUser4struct{iInt32JInt64Bbyte}typeUser5struct{JInt64BbyteIInt32}typeUser6struct{JInt64IInt32Bbyte}

According to these 6 combinations, the definition of 6 structs, respectively, bit USER1,USER2,...,USER6, then we now guess what the 6 types of structs occupy the amount of memory, is unsafe.Sizeof() the value.

You might guess 1+4+8=13, because the size of byte is 1,int32 size of 4,int64 size is 8, and struct is actually a combination of fields, so guessing the size of the struct is the sum of the field size is also very normal.

But, however, I can clearly say that this is wrong.

Why is the error, because there is memory alignment exists, the compiler uses the memory alignment, then the final size results are different. Now we formally verify the values of these kinds of structs.

123456789101112131415

func Main () {var u1 user1var u2 user2var u3 User3var u4 User4var u5 use R5var U6 user6fmt. Println ("U1 size is", unsafe. Sizeof (U1)) fmt. Println ("U2 size is", unsafe. Sizeof (U2)) Fmt. Println ("U3 size is", unsafe. Sizeof (U3)) fmt. Println ("U4 size is", unsafe. Sizeof (U4)) fmt. Println ("U5 size is", unsafe. Sizeof (U5)) fmt. Println ("U6 size is", unsafe. Sizeof (U6))}

As you can see from the output above, the result is:

123456

U1 size is 16u2 size are 24u3 size is 16u4 size is 24u5 size is 16u6 size is 16

The results come out (the result of my computer, the MAC64 bit, you may not be the same), 4 16 bytes, 2 24 bytes, not the same, but not the same, this shows:

1. Memory alignment affects the size of a struct
2. The field order of a struct affects the size of the struct

Combined with the above two points, we can tell that the different field order ultimately determines the size of the struct's memory, so sometimes a reasonable field order can reduce the overhead of memory .

Memory alignment affects the memory footprint of the struct, and now we analyze in detail why the order of the field definitions will cause the struct's memory footprint to be different.

Before we analyze, we'll look at the rules for memory alignment:

1. For a specific type, the alignment value =min (compiler default alignment value, type size sizeof length). That is, between the default set of alignment values and the memory footprint of the type, the minimum value is the alignment value for that type. My Computer defaults to 8, so the maximum value will not exceed 8.
2. After each field is aligned with the memory, the struct itself is aligned, the alignment value =min (the default alignment value, the field Maximum type length). This is also very well understood, in all fields of the struct, the length of the largest type and the default alignment value, whichever is the smallest.

The above two rules should be well understood, understanding to understand the following struct structure can be analyzed. Here again, the alignment value is also called the alignment factor, the alignment multiplier, and the number of Zimo. This means that each field's offset in memory is a multiple of the alignment value .

We know that the alignment values of Byte,int32,int64 are 1,4,8, and the memory size is also 1,4,8. So for the first struct user1 , its field order is Byte, Int32, Int64, we first use the 1th memory alignment rule for memory alignment, and its memory structure is as follows.

1	Bxxx|iiii|jjjj|jjjj

user1Type, 1th field, byte, aligned with the value 1, size 1, so placed in the memory layout of the 1th bit.

The 2nd field int32, the Alignment value 4, the size 4, so its memory offset value must be a multiple of 4, in the current user1 , it can not start from the 2nd bit, must start from the 5th bit, that is, the offset is 4. The 2,3,4 bit is populated by the compiler, typically a value of 0, also known as a memory hole. So the 5th to 8th digit is the 2nd field I.

The 3rd field, the alignment value is 8, and the size is 8. Since the user1 first two fields are already ranked 8th, the next bit offsets are exactly 8, a multiple of the 3rd field alignment values, without padding, and you can arrange the 3rd field directly, that is, from 9th to 16th for 3rd Field J.

Now that the memory is 16 bytes long after the first memory alignment rule, we begin to align with the 2nd rule of memory. According to the second rule, the default alignment value 8, the maximum type length in the field is also 8, so we find the alignment value bit 8 of the structure, our current memory length is 16, is a multiple of 8, has achieved alignment.

So to this end, user1 the size of the structure's memory footprint is 16 bytes.

Now let's analyze a user2 type, it's size is 24, just change the order of the fields I and J, it takes more than 8 bytes, let's see why? Or first use our memory 1th rule analysis.

1	Bxxx|xxxx|jjjj|jjjj|iiii

By the alignment value and its occupied size, the 1th field B offset is 0, occupies 1 bytes, and is placed on the 1th bit.

The 2nd field, J, is int64, the alignment value and size are 8, so starting at offset 8, which is the 9th to 16th bit, is J, which means that the 2nd to 8th bit is populated by the compiler.

At present, the entire memory layout has been shifted 16 bits, which is exactly the 3rd field I of the alignment value of 4 multiples, so do not fill, can be directly arranged, 17th to 20th bit is I.

Now that all the fields are aligned, the entire memory size is 1+7+8+4=20 bytes, we start using the 2nd rule of memory alignment, which is the alignment of the struct, with the default alignment value and the largest field size, to find the alignment value of the struct as 8.

Now our entire memory layout size is 20, not a multiple of 8, so we need to make a memory fill, up to a multiple of 8, the smallest is 24, so the entire memory layout after alignment is

1	Bxxx|xxxx|jjjj|jjjj|iiii|xxxx

So this is why we finally get the user2 size of 24.
Based on the above approach, we can draw the memory layout of several other structs.

User3

1	Iiii|bxxx|jjjj|jjjj

User4

1	Iiii|xxxx|jjjj|jjjj|bxxx|xxxx

User5

1	Jjjj|jjjj|bxxx|iiii

User6

1	Jjjj|jjjj|iiii|bxxx

The above gives the answer, pushed to the process you can refer to user1 and user2 try. Next we introduce through unsafe. Pointer perform memory operations and read and write memory.

"Go language Combat" reading notes, not to be continued, welcome to sweep code attention flysnow_org to the public or website http://www.flysnow.org/, the first time to see follow-up notes. If you feel helpful, share it with your friends and thank you for your support.