Golang Internals, part 4:object Files and Function Metadata

This is a creation in Article, where the information may have evolved or changed.

All Parts:part 1 | Part 2 | Part 3 | Part 4 | Part 5

Today, we'll take a closer look at the FUNC structure and discuss a few details about how garbage collection works I N Go.

This post was a continuation of "Golang Internals, part 3:the Linker and Go Object Files" and uses the same sample program . So, if you haven ' t read it, I strongly advise it, and this before moving forward.

The structure of function metadata

The main idea behind relocations should is clear from part 3. Now let's take a look at the FUNC structure of the main method:

Func: &goobj. func{    Args:    0,    Frame:   8,    Leaf:    false,    Nosplit:false,    Var:     {    },    PCSP:   goobj. data{offset:255, Size:7},    pcfile:goobj. data{offset:263, Size:3},    pcline:goobj. data{offset:267, Size:7},    PCData: {        {offset:276, size:5},    },    Funcdata: {        {            Sym:    Goobj. Symid{name: "Gclocals 3280bececceccd33cb74587feedb1f9f", version:0},         offset:0,     },     {         Sym:    Goobj. Symid{name: "Gclocals 3280bececceccd33cb74587feedb1f9f", version:0},               offset:0,           },       },       File: {" /home/adminone/temp/test.go "},   },

You can think of this structure as function metadata emitted by the compiler in the object file and used by the Go runtime . This article explains the exact format and meaning of the different, in Func. Now, I'll try to show the metadata are used in the runtime.

Inside the runtime package, this metadata was mapped on the following struct:

Type _func struct {entry   uintptr//Start pcnameoff int32   //function Nameargs  int32//In/out args Sizeframe Int32//legacy frame size; Use PCSP if POSSIBLEPCSP      int32pcfile    int32pcln      int32npcdata int32nfuncdata   Int32}

You can see this not all the information that is in the object file has been mapped directly. Some of the fields is only used by the linker. Still, the most interesting here is the pcsp, pcfile, and pcln Fields,which is used when a PR Ogram counter is translated to a stack pointer, filename, and line accordingly.

This was required, for example, when panic occurs. At that exact moment, the runtime is knows about the program counter of the current assembly instruction that have Trigge Red the panic. So, the runtime uses this counter to obtain of the current file, line number, and full stack trace. The file and line number is resolved directly, using the pcfile and pcln fields. The stack trace is resolved recursively, using PCSP.

Now that I had a program counter, the question was, how does I get a corresponding line number? To answer it, you need-through assembly code and understand how line numbers is stored in the object file:

0x001a 00026 (Test.go:4) movq$1, (SP) 0x0022 00034 (test.go:4) pcdata$0,$00x0022 00034 (test.go:4) Call,runtime.printint ( SB) 0x0027 00039 (test.go:5) addq$8,sp0x002b 00043 (test.go:5) RET,

We can see, counters from-to-inclusive correspond to line number 4 and counters from-to- Next_func tion_program_counter–1 correspond to line number 5. For space efficiency, it's enough to store the following map:

26-439-5 ...

This was almost exactly what the compiler does. The pcln field points to a particular offset in a map that corresponds to the first program counter of the Curren t function. Knowing this offset and also the offset of the first program counter of the next function, the runtime can use binary sear Ch to find the line number, the corresponds to the given program counter.

In Go, this idea is generalized. A line number or stack pointer can is mapped to a program counter, and also any integer value. This was done via the PCDATA instruction. Each time, the linker finds the following instruction:

0x0022 00034 (test.go:4) pcdata$0,$0

It doesn ' t generate any actual assembler instructions. Instead, it stores the second argument of this instruction in a map with the current program counter, while the first Argu ment indicates what maps is used. With this first argument, we can easily add new maps, which meaning are known to the compiler and runtime but are opaque to The linker.

How the garbage collector uses function metadata

The last thing this still needs to being clarified in function metadata is the funcdata array. It contains information necessary for garbage collection. Go uses a mark-and-sweep garbage collector (GC) that operates in both stages. During the first stage (mark), it traverses through all objects that is still in use and marks them as reachable. All the unmarked objects is removed during the second (sweep) stage.

So, the garbage collector starts by looking for a reachable object in several known locations, such as global variables, p Rocessor registers, stack frames, and pointers in objects, which have already been reached. However, if you think about it carefully, the looking for pointers in stack frames are far from a trivial task. So, when the runtime was performing garbage collection, how does it distinguish whether a variable in the stack is a pointe R or belongs to a non-pointer type? This is the where funcdata comes into play.

For each function, the compiler creates and the variables. One contains a bitmap vector for the arguments area of the stack frame. The other one contains a bitmap for the rest of the frame, includes all the local variables of pointer types defined I n the function. Each of these variables tells the garbage collector, where exactly in the stack frame the pointers is located, and that I Nformation is enough for it to do it job.

It is also worth mentioning this, like PCDATA, funcdata are also generated by a pseudo-go assembly Instru Ction:

0x001a 00026 (test.go:3) funcdata$0,gclocals 3280bececceccd33cb74587feedb1f9f+0 (SB)

The first argument of this instruction indicates, whether this is function data for arguments or a local variables area. The second one is actually a reference to a hidden variable that contains a GC mask.

More on Golang

In the upcoming posts, I'll tell you about the go bootstrap process, which are the key to understanding how the go runtim E works. See your in a week.

Read all parts of the Series:part 1 | Part 2 | Part 3 | Part 4 | Part 5

about the author:  sergey Matyukevich are a Cloud Engineer and Go Developer at Altoro S. With 6+ years in software engineering, he's an expert on cloud automation and designing architectures for complex clou d-based systems. An active member of the Go community, Sergey are a frequent contributor to Open-source projects, such as Ubuntu and Juju Ch Arms.