Detecting memory leaks in the go language

Let's set up the database, set up a MySQL database table, named users, with Login_name, nickname, uid, password, forbidden several fields, where UID and forbidden are of type int field, The other are varchar types, and password is the result of user password MD5, so the length is 32. The MySQL database engine we use is go-sql-driver/mysql.

Create DATABASE mytest default character set Utf8;use mytest; CREATE table users (login_name varchar), nickname varchar (2 0), UID Int (8), password char (+), Forbidden tinyint (1)); INSERT into users value (' Alex ', ' Sunday ', ' 12345678 ', ' 827ccb0eea8a706c4c34a16891f84e7b ', ' 0 ');

　　

Package Main
Import ("Database/sql" "Fmt" _ "Github.com/go-sql-driver/mysql" "OS" "Os/signal" "Runtime" "Runtime/pprof" "Strings" " Time ") Var (pid intprogname String) func init () {pid = OS. Getpid () Paths: = Strings. Split (OS. Args[0], "/") paths = strings. Split (Paths[len (Paths)-1], string (OS. PathSeparator)) Progname = Paths[len (Paths) -1]runtime. Memprofilerate = 1}func Saveheapprofile () {runtime. GC () F, err: = OS. Create (FMT. Sprintf ("Heap_%s_%d_%s.prof", Progname, PID, time. Now (). Format ("2006_01_02_03_04_05"))) if err! = Nil {return}defer f.close () pprof. Lookup ("Heap"). WriteTo (f, 1)}func waitforsignal () os. Signal {Signalchan: = Make (chan os. Signal, 1) defer close (Signalchan) Signal. Notify (Signalchan, OS. Kill, OS. Interrupt) S: = <-signalchansignal.stop (Signalchan) return S}func Connect (source string) *sql. DB {db, err: = SQL. Open ("MySQL", source) if err! = Nil {return nil}if err: = db. Ping (); Err! = Nil {return Nil}return db}type User struct {uid intname stringnick stringforbidden intcid int }funC Query (DB *sql. DB, name string, id int, Datachan chan *user) {for {time. Sleep (time.millisecond) User: = &user{cid:id,name:name,}err: = db. Queryrow ("Select Nickname, uid, forbidden from users WHERE login_name =?", name). Scan (&user.nick, &user.uid, &user.forbidden) if err! = Nil {Continue}datachan <-user}}func main () {defer s Aveheapprofile () DB: = Connect ("mytest:[email protected" (localhost:3306)/mytest?charset=utf8 ") if db = = Nil { Return}userchan: = Make (chan *user, max) for I: = 0; I < 100; i++ {Go query (db, "Alex", I+1, Userchan)}allusers: = Make ([]*user, 1<<12) go func () {for User: = Range Userchan {fmt. Printf ("routine[%d] get user%+v\n", user.cid, user) AllUsers = append (allUsers, User)}} () s: = Waitforsignal () fmt. PRINTF ("Signal got:%v, all Users:%d\n", S, Len (allUsers))}

　　

The above program of course has a pretty serious memory leak problem, let's see how to add code, let pprof to help us locate the memory leak in the specific code snippet.

The following is a general procedure for diagnosing memory leak problems:

1. Under the command line go build compiles an executable program, for example called Your-executable-name, and then runs it to run (it will keep running until you actively interrupt, the system will monitor the interrupt signal and write the heap information to the generated file We call it a profile-filename)
2. Use to go tool pprof your-executable-name profile-filename enter PPROF command mode to analyze data
3. Or use go tool pprof your-executable-name --text profile-filename the view the memory consumption rank of each function/method
4. or use a go tool pprof your-executable-name --dot profile-filename > heap.gv command to generate a GV file that can be seen in Graphviz, viewing the memory consumption of each method/function while looking at the call relationship between them
5. or after generating a GV file by dot -Tpng heap.gv > heap.png generating a PNG graphic file that calls network and memory consumption graphs

Subsequent execution go tool pprof your-executable-name --text profile-filename results in a similar result (only the first few lines are intercepted):


Adjusting heap profiles for 1-in-1 sampling rate
Total: 1.7 MB
0.7 40.4% 40.4% 1.0 56.2% github.com/go-sql-driver/mysql.(*MySQLDriver).Open
0.5 27.7% 68.1% 1.6 93.6% main.query
0.2 11.7% 79.8% 0.2 11.7% newdefer
0.1 6.9% 86.7% 0.1 6.9% database/sql.convertAssign
0.1 4.6% 91.3% 0.1 4.7% main.func路001
0.0 1.2% 92.5% 0.0 1.2% net.newFD
0.0 1.0% 93.5% 0.0 1.0% github.com/go-sql-driver/mysql.parseDSNParams
0.0 0.9% 94.5% 0.0 0.9% runtime.malg
0.0 0.6% 95.1% 0.0 0.6% runtime.allocm
0.0 0.5% 95.6% 0.0 0.5% resizefintab
0.0 0.5% 96.1% 0.0 0.5% github.com/go-sql-driver/mysql.(*mysqlConn).readColumns
0.0 0.5% 96.6% 0.0 0.5% database/sql.(*DB).addDepLocked


The meaning of each column in this table is in the Perftool document.

Run the go tool pprof command, without the –-text parameter will go directly into the pprof command line mode, you can first execute TOP10, you can get a ranking similar to the above results,

From the inside you can see that the most consumed memory is the MySQL open method, indicating that we have called the Open method after the release of resources.

In addition we can also run go tool pprof your-executable-name --dot profile-filename > heap.gv , so we will get a HEAP.GV file, we open this file in Graphviz will get a more detailed memory consumption diagram including the call relationship. Of course, if we only need a picture, we can also run to dot -Tpng heap.gv > heap.png save the GV file as a PNG, so that I can show the results of the analysis below.

In addition to printing out memory profiling information to a file at a given moment, if you want to be able to see the results of the analysis at any time, there can be a very simple way to get the net/http and net/http/pprof into the import, which net/http/ The Pprof package needs to be imported in a import _ "net/http/pprof" way that adds a custom port (such as 6789) to the HTTP server in the code, like this:

Go func () {    http. Listenandserve (": 6789", Nil)} ()

　　

This way, after the program has run, you can go tool pprof your-executable-name http://localhost:6789/debug/pprof/heap get real-time memory profiling information, the data format is consistent with the format saved by the file, and then the processing will be the same.

After Go tool pprof, enter PPROF command line mode, you can use the List command to see the corresponding function (actually matches the function name of the regular expression) in the specific row resulting in the performance/memory loss.

Transferred from: http://blog.raphaelzhang.com/2014/01/memory-leak-detection-in-go/

Detecting memory leaks in the go language