How do I do performance optimizations for go?

This is a creation in Article, where the information may have evolved or changed.

Go performance optimizations are generally similar to those of C + +, but also have its own unique troubleshooting methods and pitfalls, all derived from its language features and environment.

1. Performance optimization premise--Any good thing is on the right premise

Many things in the code world are closely related to the philosophy of our life, we want to do a good job, first of all to ensure that we can finish our task on time, and then think about how to do the work better. If blindly only to ask for perfection may lead to delay, failure, halfway.

So, write the correct code first, then consider how to make the code run faster and better, complete the basic functionality, and then figure out how to optimize it. The right is the basis of optimization, without this foundation, any optimization is meaningless.

2. Performance optimization limits--architecture design and hardware Resources

Good architecture design is the premise that we can perform performance, and a poorly designed architecture pays more attention to optimize the effect is also greatly discounted. This is why we often see that as the volume of business or the increase in the number of users of the architecture will continue to evolve and change, if the beginning of the design of the architecture can be sustained, then please accept my knees!

Better understanding of hardware resources, a 16-core, 64G memory server and 4-core, 4G memory of the garbage machine comparison is almost heaven and land. Needless to say.

3. When to do performance optimization

We should forget about small efficiencies, say about 97% of the time; Premature optimization is the root of all evil (about 97% of the time, we should forget the small optimizations that premature optimizations are the root of all evil). --donald E.knuth

This sentence is not to say not to optimize, not to think about the algorithm, but in the early days we should focus more on the implementation of the program, rather than the beginning to think about optimization, you can open to write. Slowly there will be a driving force that we do not consciously to optimize.

Under normal circumstances this kind of drive I think there are two, one is self-driven, such as the experience of the ACM or algorithmic competition in the children's shoes in the face of a problem will not consciously from the complexity of the analysis of the problem, or an "obsessive-compulsive disorder" can not tolerate slow, card, collapse and so on.

The other is environment-driven, such as high concurrency environment, high-precision environment, low latency environment, big Data environment and so on some aspect of our system and even many aspects have stringent requirements, forcing this we need to continue to excellent (Jia) (ban), excellent (Jia) (ban), and then Excellent (Jia) (ban).

• When you realize that this function may be called frequently, you need to find a way to optimize
• When you realize that this data structure is not properly designed to make memory consumption too high, you need to find ways to optimize
• When a user reflects a service response that is too slow, it needs to optimize
• When a boss has a good service and doesn't want to spend money on machines, he needs to optimize
• When the code is too messy, a lot of problems, often reported warning disturb and female ticket play, you need to optimize
• 。。。

4. How long does it take to optimize performance?

Some people say that is 28 law, also known as 80/20 Law, Pareto Law (law) also called Barnett Law, the most labor-saving law, imbalance principle, etc., is widely used in sociology and enterprise management. It was discovered by Pareto, an Italian economist at the end of 19th century, in early 20th century. He thinks that in any group of things, the most important is only a small part, about 20%, the remaining 80%, although the majority, but is secondary, and therefore called the 28 law. --Baidu Encyclopedia

I think that although we do not have to follow 2:8 of the requirements of the implementation, but there is no doubt that the optimization will cost us a lot of time and energy, and far greater than the time of our system implementation, or since the first time the development, all the time is to do optimization. Own experience, at that time in order to give a bank to do 60W Terminal testing optimization of an API cache system, the basic ability to achieve two weeks to complete, I and performance QA children's shoes after the optimization-test-optimization-test, spent one months to do this thing, this is not over, Back in the real environment testing process still exposed a lot of problems, such as goruntine explosion backlog, CPU explosion and so on, later found to be architectural design and component use problems, yes, when this problem is not can not be solved, but in order to solve such problems will make the system complex, bloated, Although the development experience is not much, but I think that is the code implementation of the code implementation, the component solves the problem should be the component to solve, the architecture design problem is that the architecture needs to be improved, do not say all can be solved in the code, unless it is the last resort.

5. 工欲善其事, its prerequisite

The first is the code hierarchy, good code is the performance of the key factors, how to achieve the efficiency of the function, ranking is not efficient, high concurrency and so on, you can use the Code Quality Assessment tool to do the evaluation, of course, it is best to let experienced drivers hands-on guidance.

Go Code evaluation Tool:

• Goreporter – Generate Go Code quality assessment Report
• dingo-hunter – A static analyzer for finding deadlocks in Go programs
• Flen – Get function length information in the Go package
• go/ast –package AST declares the type of go package used to represent the syntax tree
• Gocyclo – Measuring the complexity of cyclomatic functions in Go source code
• go Meta linter – Simultaneous Go lint tool and output standardization of tools
• Go vet – detects the go source code and reports suspicious constructs
• ineffassign – Detection of invalid assignments in Go code
• safesql –golang Static analysis tool to prevent SQL injection

Then how to debug the Go program during the run, go comes with a pprof tool that can do CPU and memory profiling. Use can refer to the previous article: performance optimization of an internal API system-the column of knowledge

package mainimport   (    "log"    "net/http"   _"net/http/pprof"   )func main()  {      go func() {        //port is you coustom define.        log.Println(http.ListenAndServe("localhost:7000", nil))      }()      //do your stuff.}

Only need to introduce net/http and _ "Net/http/pprof", and then with the tool to generate a flowchart, the proportion of the chart is clear.

Or for some programs you can also change it at run time, debugging it, using Google/gops Google production, you can go to view stacks, memory, cpu,heap and so on information, very good, but I do not like it to open the service port, This project just started with no need to use the new port, directly using the socket file communication, but because it can not be implemented on Windows, finally forget, since the goodwill down!

Of course you can also use the GDB tool, the latest GDB seems to have joined the command to view Goruntine, great!

6. Algorithms and optimization ideas

This needless to say, to tell the truth, the algorithm is to distinguish between engineers and yards of a very large demarcation point, the algorithm can be said to be the basic ability, many people do not think it is only the interview threshold, but look at the code implementation of the data structure design and algorithm implementation will understand! When encountering a problem will not consciously think of an algorithm, this purpose is enough, in fact, did not say that the algorithm is very good, in fact, before the old driver chat also said as long as you can think of the problem can be solved with what algorithm.

Various sorting, collection operations, queries, etc., no best algorithms, as long as the most suitable algorithm .

As to which aspects need optimization, on the one hand is the efficiency of the algorithm is also the phenomenon, such as the CPU is particularly high so look at the Goruntine scheduling, which function occupies higher than the existence of a dead loop, memory large, to see if there is a large memory allocation is not timely recovery, or is not a frequent memory allocation, Is there a memory leak? Slow response is where the card is, whether it is performing efficiently or communicating with components, and so on.

7.Go Traps and Tricks

A.make's Trap

func main() {s := make([]int, 3)s = append(s, 1, 2, 3)fmt.Println(s)}结果[0 0 0 1 2 3]

B.map Read and write conflicts, producing race state

C. File open, database connection remember to close or release, general use defer

D. For a map of a struct value, you cannot update a single struct value

E. Simplifying range

forrangem{}

F.defer's Trap

A well-known return value is declared at the same time as the function declaration, and the anonymous return value is declared when the return is executed, so only the named return value can be accessed in the defer statement, and the anonymous return value cannot be accessed directly.

package mainimport ("fmt")func main() {fmt.Println("return:", defer_call())}func defer_call() int {var i intdefer func() {i++fmt.Println("defer1:", i)}()defer func() {i++fmt.Println("defer2:", i)}()return i}defer2: 1defer1: 2return: 0

Q2.

package mainimport ("fmt")func main() {fmt.Println("return:", defer_call())}func defer_call() (i int) {defer func() {i++fmt.Println("defer1:", i)}()defer func() {i++fmt.Println("defer2:", i)}()return i}defer2: 1defer1: 2return: 2

G. Pitfalls of short-variant declarations

Developers who have used dynamic languages are familiar with the syntax of short-variant declarations, so it's easy to think of them as a normal allocation operation. This error, there will be no compilation errors, but will not achieve the effect you expect.

package mainimport "fmt"func main() {      value := 1    fmt.Println(value)     // prints 1    {        fmt.Println(value) // prints 1        value := 2        fmt.Println(value) // prints 2    }    fmt.Println(value)     // prints 1 (bad if you need 2)}

The bottom line is that the code boundaries and variables affect the scope of the problem.

H.nil and Explicit types

The nil glyph is used to denote "0 values" for interface, functions, maps, slices, and channels. If you do not specify the type of the variable, the compiler will not be able to compile your code because it does not know the exact type, and you cannot assign a nil value to string.

package mainfunc main() {    var value1 = nil // error    _ = value1    var value2 string = nil // error    if value2 == nil { // error        value2 = "test"    }}

Should

package mainfunc main() {var value1 interface{} = nil // error_ = value1var value2 stringif value2 == "" {value2 = "test"}}

I. All values are passed, no reference is passed

If you are a C or C + + developer, you know that an array is a pointer. When you pass an array to a function, the function references the same area of memory so that they can modify the original data. But the array in Go is numeric, so when you pass an array to the function, the function gets a copy of the original array data. If you are going to update the data in the array, you will fail.

All case traversal under J.select is random, and in the process of use, be aware that this is different from switch

L. Implement a Type classification function using an interface:

func classifier(items ...interface{}) {    for i, x := range items {        switch x.(type) {        case bool:            fmt.Printf("param #%d is a bool\n", i)        case float64:            fmt.Printf("param #%d is a float64\n", i)        case int, int64:            fmt.Printf("param #%d is an int\n", i)        case nil:            fmt.Printf("param #%d is nil\n", i)        case string:            fmt.Printf("param #%d is a string\n", i)        default:            fmt.Printf("param #%d’s type is unknown\n", i)        }    }}

The L.map value is unordered when it gets, so it needs to be sorted indirectly by string array when we need to order.

package mainimport (    "fmt"    "sort")func main() {    var m = map[string]int{        "unix":         0,        "python":       1,        "go":           2,        "javascript":   3,        "testing":      4,        "philosophy":   5,        "startups":     6,        "productivity": 7,        "hn":           8,        "reddit":       9,        "C++":          10,    }    var keys []string    for k := range m {        keys = append(keys, k)    }    sort.Strings(keys)    for _, k := range keys {        fmt.Println("Key:", k, "Value:", m[k])    }}

M.init function

Development process We often encounter the main logic before the start to declare or some global variables or initialization operations, or sometimes we just need to import some packages, do not need to use the function inside, it is necessary to use the Init initialization function, a package can have more than one init, For example, if you have an init in demo/a.go,demo/b.go, then they will execute.

The N.GO program shows that memory consumption is sometimes not really in use, but it has not yet been returned to the operating system

O. The study of the rain mark teacher

Rain Scar Academy-Segmentfault

P.golang 50-degree ash

Chinese: Go 50 Gray: Golang pitfalls and common mistakes to be noticed by new developers

English version: Shades of Go:traps, gotchas, and Common mistakes for New Golang Devs

Follow-up replenishment ...