Golang performance analysis using PPROF and Go-torch

In the process of software development, project launch is not the end point. After the line, but also to the sampling analysis of the operation of the program, and re-construct the existing functions, so that the program execution more efficient and more stable writing. Golang's toolkit comes with pprof function, which makes it easy to find out some of the functions of memory and CPU in the program. With Uber's flame diagram and visual display, we're more straightforward when it comes to analyzing programs.

Pprof has two packages to analyze the program one is net/http/pprof and the other is runtime/pprof,net/http/pprof just encapsulates the RUNTIME/PPROF package and exposes it with HTTP, as shown in the source code:

Image

Using NET/HTTP/PPROF to analyze Web services

Pprof Analysis Web project, very simple only need to import the package.

_ "net/http/pprof"

Write a small Web server

Package Mainimport (_ "net/http/pprof" "Net/http" "Time" "Math/rand" "FMT") var Count int64 = 0func Main () {Go Calcount () http. Handlefunc ("/test", test) http. Handlefunc ("/data", handlerdata) Err: = http. Listenandserve (": 9909", nil) if err! = Nil {panic (err)}}func Handlerdata (w http. Responsewriter, R *http. Request) {qurl: = R.url fmt. Println (qurl) Fibrev: = Fib () var fib uint64 for i:= 0; I < 5000; i++ {fib = Fibrev () fmt. Println ("fib =", fib)} str: = Randomstr (Randomint (+)) str = FMT. Sprintf ("Fib =%d; String =%s ", fib, str) w.write ([]byte (str))}func Test (w http. Responsewriter, R *http. Request) {fibrev: = fib () var fib uint64 index: = Count arr: = Make ([]uint64, index) var i int64 for; I < index; i++ {fib = Fibrev () arr[i] = fib FMT. Println ("fib =", fib)} time. Sleep (Time.millisecond *) str: = FMT. Sprintf ("Fib =%v", arr) W.        Write ([]byte (str))}func Fib () func () UInt64 {var x, y UInt64 = 0, 1 return func () UInt64 {x, y = y, x + y return x}}var letterrunes = []rune ("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890") func randoms TR (num int) string {seed: = time. Now (). Unixnano () If seed <= 0 {seed = time. Now (). Unixnano ()} rand. Seed (seed) B: = make ([]rune, num) for I: = range b {b[i] = Letterrunes[rand. INTN (Len (Letterrunes))]} return string (b)}func randomint (min, max int) int {rand. Seed (time. Now (). Unixnano ()) return Rand. INTN (max-min + 1) + Min}func Calcount () {timeinterval: = time. Tick (time. Second) for {select {case i: = <-Timeinterval:count = Int64 (I.second ())}}}

Web Services listening on port 9909

Web server has two HTTP methods
Test: Fibonacci calculations based on the current number of seconds
Data: Make a 5000 Fibonacci calculation and return a random string

Run the program to view profiles information about the Web version by accessing http://192.168.3.34:9909/debug/pprof/

Image

These paths represent the

/debug/pprof/profile: Access to this link will automatically be CPU profiling, continuous 30s, and generate a file for download

/debug/pprof/block:goroutine the record of the blocking event. The default is sampled once for each blocking event that occurs.

/debug/pprof/goroutines: A record of active goroutine information. Sampled only once on acquisition.

/debug/pprof/heap: A record of the heap memory allocation situation. The default is sampled once per 512K bytes allocated.

/debug/pprof/mutex: View the holder of the contention mutex.

/debug/pprof/threadcreate: Record of system thread creation. Sampled only once on acquisition.

In addition to these golang provide me with more convenient methods for analysis, let's use the command to access detailed information

We use WRK to access our two methods so that our service will be at a high speed and the results of the sampling will be more accurate.

wrk -c 20 -t 5 -d 3m http://192.168.3.34:9909/datawrk -c 20 -t 5 -d 3m http://192.168.3.34:9909/test

Analyzing CPU Usage

Using commands to analyze CPU usage

<pre style= "margin:0px; padding:0px; White-space:pre-wrap; Word-wrap:break-word; font-family: "Courier New"!important; FONT-SIZE:12PX!important; " >go tool Pprof Httpdemo http://192.168.3.34:9909/debug/pprof/profile</pre>

By default, the run-time system of the Go language samples CPU usage at a frequency of. That is, the sample is sampled 100 times per second, that is, every 10 milliseconds. Why use this frequency? Because it's enough to produce useful data, it doesn't stop the system from stalling. and 100 of this number is also easy to do conversion, such as gross position sampling count to the number of samples per second. In fact, the sampling of CPU usage described here is a sampling of the program counters on the current goroutine stack.

The default sampling time is 30s you can specify the sampling time by using the-seconds command. The command line status is entered when the sample is completed:

Image

You can enter help to view related commands. Here's a few common commands.

Top command, entering the top command defaults to the top 10 CPU-intensive method. Of course, people can add numbers to the top number after the command.

Image

The list command outputs the relevant method according to your regular output. The direct and optional o outputs all methods. You can also specify the method name

Image

Example: The Handlerdata method accounts for 74.81% of the CPU

Web commands: Show as a Web page: More intuitive display of CPU usage

Image

Analyze Memory usage

and analysis CPU almost use command

go tool pprof httpdemo http://192.168.3.34:9909/debug/pprof/heap

By default, when sampling only the current memory usage, you can add optional command alloc_objects to sample the memory from the beginning of the program

go tool pprof -alloc_objects httpdemo http://192.168.3.34:9909/debug/pprof/heap

As with the CPU commands, the top list Web. The difference is that memory usage is shown here. I'm not going to show you here.

Image

Installing Go-torch

and a more convenient tool is the go-torch of Uber.

Installation is simple

go get github.com/uber/go-torchcd $GOPATH/src/github.com/uber/go-torchgit clone https://github.com/brendangregg/FlameGraph.git

Then run the copy under Flamegraph flamegraph.pl to/usr/local/bin

Flame Diagram Analysis CPU

Using commands

go-torch -u http://192.168.3.34:9909  --seconds 60 -f cpu.svg

Generates the Cpu.svg file in the current directory and opens it using the browser

Image

More intuitive to see the application's problems. The Handlerdata method takes too long CPU time. Then it goes to the code to analyze and optimize.

Flame Diagram Analysis Memory

Using commands

go-torch  http://192.168.3.34:9909/debug/pprof/heap --colors mem  -f mem.svg

Generates the Cpu.svg file in the current directory and opens it using the browser

Image

Using RUNTIME/PPROF to analyze projects

If your project is not a Web service, such as an RPC service, you should use Runtime/pprof. He offers a lot of ways to have time to look at the source code

Image

I have written a simple tool class. Used to invoke the analysis

Import ("OS" "Rrnc_im/lib/zaplogger" "Go.uber.org/zap" "Runtime/pprof" "Runtime") func Startcpuprof () {f, ERR: = OS. Create ("Cpu.prof") if err! = Nil {Zaplogger. Error ("Create CPU Profile File Error:", Zap.) Error (ERR)) return} if Err: = Pprof. Startcpuprofile (f); Err! = Nil {Zaplogger. Error ("Can not start CPU profile, Error:", Zap.) Error (Err)) F.close ()}}func stopcpuprof () {pprof. Stopcpuprofile ()}//--------Memfunc PROFGC () {runtime. GC ()//Get up-to-date Statistics}func savememprof () {f, err: = OS. Create ("Mem.prof") if err! = Nil {Zaplogger. Error ("Create Mem Profile File error:", Zap.) Error (ERR)) return} if Err: = Pprof. Writeheapprofile (f); Err! = Nil {Zaplogger. Error ("Could not write memory profile:", Zap.) Error (Err)} f.close ()}//Goroutine Blockfunc Saveblockprofile () {f, err: = OS. Create ("Block.prof") if err! = Nil {Zaplogger. Error ("Create Mem Profile File error:", Zap.) Error (ERR)) return} If err: = Pprof. Lookup ("block"). WriteTo (f, 0); Err! = Nil {Zaplogger. Error ("Could not write Block profile:", Zap.) Error (Err))} f.close ()}

Call these methods within the method you need to analyze, like I've opened up several methods with RPC.

type TestProf struct {}func (*TestProf) StartCpuProAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {    profapp.StartCpuProf() return nil}func (*TestProf) StopCpuProfAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {    profapp.StopCpuProf() return nil}func (*TestProf) ProfGcAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {    profapp.ProfGc() return nil}func (*TestProf) SaveMemAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {    profapp.SaveMemProf() return nil}func (*TestProf) SaveBlockProfileAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {    profapp.SaveBlockProfile() return nil}

Call

profTest.StartCpuProAct(context.TODO(), &im_test.TestRequest{})    time.Sleep(time.Second * 30)    profTest.StopCpuProfAct(context.TODO(), &im_test.TestRequest{})    profTest.SaveMemAct(context.TODO(), &im_test.TestRequest{})    profTest.SaveBlockProfileAct(context.TODO(), &im_test.TestRequest{})

The idea is the same, the profile is exported within the current folder. Then use the flame diagram to analyze, you can not specify the domain name, to specify the file

go-torch  httpdemo cpu.prof go-torch  httpdemo mem.prof

Reproduced in:
Author: Li Peng

Source: http://www.cnblogs.com/li-peng/

This article is copyright to the author and the blog Park, Welcome to reprint, but without the consent of the author must retain this paragraph, and in the article page obvious location to the original link, otherwise reserves the right to pursue legal responsibility.