Performance optimization and pprof use of Go programs

This is a creation in Article, where the information may have evolved or changed.

Performance optimization and pprof use of Go programs

The performance optimization of the program is nothing more than the optimization of the program occupation resources. For servers, the two most important resources are CPU and memory. Performance optimization, in the case of not affecting the program data processing capabilities, we usually require the program's CPU memory footprint as low as possible. Conversely, when the program CPU and memory consumption is not changed, as far as possible to improve the program's data processing capacity or throughput.

The native toolchain of Go provides a lot of tools for developers to use, including pprof.

The use of pprof is divided into the following two parts.

WEB programs using Pprof

Write a simple Web service program first. The program receives the request on port 9876.

 PackageMainImport("bytes"    "Io/ioutil"    "Log"    "Math/rand"    "Net/http"_"Net/http/pprof")funcMain () {http. Handlefunc ("/test", handler) log. Fatal (http. Listenandserve (": 9876",Nil))}funcHandler (W http. Responsewriter, R *http. Request) {err: = R.parseform ()if Nil! = Err {w.write ([]byte(Err. Error ()))return} dosomethingone (10000Buff: = Gensomebytes () b, err: = Ioutil. ReadAll (Buff)if Nil! = Err {w.write ([]byte(Err. Error ()))return} w.write (b)}funcDosomethingone (Timesint) { forI: =0; I < times; i++ { forJ: =0; J < Times; J + + {}}}funcGensomebytes () *bytes. Buffer {varBuff bytes. Buffer forI: =1; I <20000; i++ {buff. Write ([]byte{'0' +byte(Rand. INTN (Ten))})    }return&buff}

You can see that we simply introduced net/http/pprof , not shown to use.

Start the program.

We use it wrk to simply simulate the request.

wrk -c 400 -t 8 -d 3m http://localhost:9876/test

When we open http://localhost:9876/debug/pprof , we will show the following page:

Users can click the appropriate link to browse the content. But that's not what we're talking about, and it doesn't look intuitive.

We open the link http://localhost:9876/debug/pprof/profile a moment later and can download to the file profile .

Open with the Pprof tool that comes with Go. go tool pprof test profile. (proof followed by test for the program compiled executable file)

The input top command gets:

You can see that the CPU consumes the top 10 functions, and we can optimize this analysis.

But it may not be intuitive.

We enter the command web (required to install Graphviz,macos brew install graphviz ), the interface will open in the browser as follows:

You can see that Main.dosomethingone consumes 92.46% of the CPU time and needs to be optimized.

The Web-style CPU time graph is fully sufficient for optimization, and here we introduce the generation of the flame diagram. MacOS recommends using go-torch tools. Use methods and go tool pprof similarities.

go-torch test profileThe Torch.svg file is generated. Can be opened with a browser.

Just talked about the CPU occupancy analysis file Generation view, in fact, the generation of memory snapshots similar. http://localhost:9876/debug/pprof/heap, the file will be downloaded heap.gz .

We can also use it go tool pprof test heap.gz , and then enter top or web command to view the relevant content.

Generic program using Pprof

We write the Go program is not all WEB programs, then use the above method is not.

We can still use the pprof tool, but the location is introduced runtime/pprof .

Two functions are posted here as an example:

//Generate CPU reportsfuncCpuprofile () {f, err: = OS. OpenFile ("Cpu.prof", OS. O_rdwr|os. O_create,0644)ifErr! =Nil{log. Fatal (ERR)}deferF.close () log. Println ("CPU profile Started") Pprof. Startcpuprofile (f)deferPprof. Stopcpuprofile () time. Sleep ( -* Time. Second) fmt. Println ("CPU profile stopped")}//Generate heap Memory reportfuncHeapprofile () {f, err: = OS. OpenFile ("Heap.prof", OS. O_rdwr|os. O_create,0644)ifErr! =Nil{log. Fatal (ERR)}deferF.close () time. Sleep ( -* Time. Second) pprof. Writeheapprofile (f) fmt. Println ("Heap profile generated")}

Two functions are generated and files are created separately cpu.prof heap.prof . You can still use the go tool pprof tool for analysis, not to repeat it here.

Trace Report

Direct Sticker Code:

// 生成追踪报告func traceProfile() {    f, err := os.OpenFile("trace.out"0644)    ifnil {        log.Fatal(err)    }    defer f.Close()    log.Println("Trace started")    trace.Start(f)    defer trace.Stop()    time.Sleep(60 * time.Second)    fmt.Println("Trace stopped")}

Using the tool go tool trace for analysis, you get a very detailed tracking report for more in-depth program analysis optimization. Due to the complexity of the report, and the use of similar methods, it will not continue. Readers can try it on their own.

Paste a map of the internet to give you a look at:

Reference: Https://github.com/caibirdme/hand-to-hand-optimize-go