Debug core dumps of the Go language

This is a creation in Article, where the information may have evolved or changed.

Translate the original link reprint/reprint please indicate the source

English original link "go, the Unwritten parts" published in 2017/05/22 author JBD is a member of the Go Language development team

Checking the execution path and current state of the program is a useful debugging tool. The core file contains a memory dump and state of a running process. It is primarily used as a post-mortem debug program. It can also be used to view the status of a running program. These two usage scenarios make debugging file dumps a very good diagnostic tool. We can use this method to do the post-mortem diagnosis and analysis of the online service (production services).

In this article, we will use a simple Hello World Web service as an example. In reality, our programs can easily become complex. Analyzing a core dump gives us an opportunity to refactor the state of the program and view cases that can only be reproduced in certain conditions/environments.

Author Note : This debugging process is only possible on Linux. I'm not really sure if it works on other unixs systems. MacOS doesn't support this yet. Windows is not currently supported.

Before we start, we need to make sure that the ulimit of the core dump is set to the right range. Its default value is 0, which means that the maximum core file size is 0. I usually set it to unlimited on my development machine. Use the following command:

$ ulimit -c unlimited

Next, you need to install the delve on your machine.

Below we use the main.go file. It registers a simple request handler function (handler) and then initiates the HTTP service.

$ cat main.gopackage mainimport (    "fmt"    "log"    "net/http")func main() {    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {        fmt.Fprint(w, "hello world\n")    })    log.Fatal(http.ListenAndServe("localhost:7777", nil))}

Let's compile and produce a binary file.

$ go build .

Now let's assume that there are some problems with this server, but we are not sure about the root cause of the problem. You may have added a lot of ancillary information to your program, but you still can't find clues from these debug messages. Typically, a snapshot of the current process is useful in this case. We can use this snapshot to drill down into the current state of the program.

There are several ways to get the core file. You may already be familiar with the crash dump (crash dumps). They are the core dumps that are written to disk when a program crashes. The go language does not produce a crash dump under the default settings. But when you GOTRACEBACK set the environment variable to "crash", you can use it to trigger a crash Ctrl+backslash dump. As shown in the following:

$ GOTRACEBACK=crash ./hello(Ctrl+\)

The action above causes the program to terminate, print the stack trace, and write the core dump file to disk.

Another method is to obtain a core dump from a running program without terminating the corresponding process. The gcore core files can be produced without having to exit the running program.

$ ./hello &$ gcore 546 # 546 is the PID of hello.

Based on the above operation, we have obtained a dump without terminating the corresponding process. The next step is to load the core file into the delve and start analyzing it.

$ dlv core ./hello core.546

That's almost all. Common operations for delve are available. You can backtrace,list, view variables, and so on. Some features are not available because the core dump that we use is a snapshot rather than a running process. However, the program execution path and state are all accessible.

(DLV) bt 0 0x0000000000457774 in runtime.raise at/usr/lib/go/src/runtime/sys_linux_amd64.s:110 1 0X000000000043F7FB In Runtime.diefromsignal at/usr/lib/go/src/runtime/signal_unix.go:323 2 0x000000000043f9a1 in Runtime.crash at/us r/lib/go/src/runtime/signal_unix.go:409 3 0x000000000043e982 in Runtime.sighandler at/usr/lib/go/src/runtime/signal _sighandler.go:129 4 0x000000000043f2d1 in Runtime.sigtrampgo at/usr/lib/go/src/runtime/signal_unix.go:257 5 0x00000 000004579d3 in Runtime.sigtramp at/usr/lib/go/src/runtime/sys_linux_amd64.s:262 6 0x00007ff68afec330 in (nil) at: 0 7 0X000000000040F2D6 in Runtime.notetsleep at/usr/lib/go/src/runtime/lock_futex.go:209 8 0x0000000000435be5 in Run Time.sysmon at/usr/lib/go/src/runtime/proc.go:3866 9 0x000000000042ee2e in Runtime.mstart1 at/usr/lib/go/src/runt ime/proc.go:118210 0x000000000042ed04 in Runtime.mstart at/usr/lib/go/src/runtime/proc.go:1152 (DLV) ls> RUNTIME.R Aise ()/usr/lib/go/src/runtime/sys_linux_amd64.s:110 (pc:0x457774) 105:syscall 106:movl AX, DI//arg 1 tid 107:movl sig  +0 (FP), SI//ARG 2 108:MOVL, AX//Syscall-tkill 109:syscall=> 110:ret 111:112:    TEXT runtime Raiseproc (SB), nosplit,$0 113:MOVL, AX//Syscall-getpid 114:syscall 115:MOVL AX, DI//ARG 1 PID