Design and implementation of Golang multi-level memory pool

design and implementation of Golang multi-level memory pool

Last month, toothpaste factory Intel was patched for CPU firmware and systems because of two bugs in meltdown and Spectre. Our production environment uses Alibaba Cloud, after patching, several IO-intensive machine performance decreased significantly, from the traffic and CPU load estimates, the performance impact of about 50%, not a good deal to reduce the maximum of 30%.

In the run business is go write, use go pprof to program profiling, inadvertently found that the current system GC and malloc high. Which Ioutil. The readall takes up considerable CPU time.
Ioutil. Why is ReadAll slow?

The signature prototype of this function is the Func readall (r io. Reader) ([]byte, error). The team's small partners like to use this function, one of the reasons is that the function can read the data in R once and return it, do not need to care about how the memory is allocated, if the allocated memory is not enough, how to expand memory and so on. As a util function, this design is completely fine. However, in IO-intensive scenarios, the overhead of this function is what you need to be concerned about. This function actually calls Realall to read the data:

ReadAll reads from R until an error or EOF and returns the data it read
From the internal buffer allocated with a specified capacity.
Func ReadAll (R io. Reader, capacity Int64) (b []byte, err Error) {
BUF: = bytes. Newbuffer (Make ([]byte, 0, capacity))
If the buffer overflows, we'll get bytes. Errtoolarge.
Return as an error. Any other panic remains.
Defer func () {
E: = Recover ()
If E = = Nil {
Return
}
If panicerr, OK: = E. (error); Ok && panicerr = = bytes. Errtoolarge {
Err = Panicerr
} else {
Panic (e)
}
}()
_, Err = buf. Readfrom (R)
Return BUF. Bytes (), err
}

Where capacity is a constant value of 512. The Realall function reads the data in the call Buf.readfrom:

Readfrom reads data from R until EOF and appends it to the buffer, growing
The buffer as needed. The return value n is the number of bytes read. Any
Error except IO. EOF encountered during the read is also returned. If the
Buffer becomes too large, readfrom'll panic with Errtoolarge.
Func (b *buffer) readfrom (R io. Reader) (n Int64, err error) {
B.lastread = Opinvalid
If buffer is empty and reset to recover space.
If B.off >= len (b.buf) {
B.reset ()
}
for {
If free: = Cap (B.BUF)-Len (B.BUF); Free < Minread {
Not enough space at end
Newbuf: = B.buf
If B.off+free < Minread {
Not enough space using beginning of buffer;
Double buffer capacity
Newbuf = Makeslice (2*cap (b.buf) + minread)//1 Expansion memory
}
Copy (Newbuf, B.buf[b.off:])//2 copy Content
B.buf = Newbuf[:len (b.buf)-b.off]
B.off = 0
}
M, E: = R.read (B.buf[len (b.buf): Cap (B.BUF)])
B.buf = B.buf[0:len (b.buf) +m]
n + = Int64 (m)
If E = = io. EOF {
Break
}
If E! = Nil {
return N, E
}
}
return n, nil//err is EOF, so return nil explicitly
}

The reason for this is clear: if the size of the data to be read exceeds the initial buf size (the default initial size is bytes), the memory is redistributed and the contents are copied to the new buffer. If the data to be read is very large, the above actions are repeated several times. Then the problem of optimization translates into how to reduce memory redistribution and copying. design and implementation of multi-level memory pool

1. The memory pool is divided into multiple levels according to size. As shown above, (0, 1024] Use level 0, (1024, 2048] use the 1. There are two benefits of a memory pool rating:

A. Flexibility to plan the total size and item count of different levels of memory pools to suit different businesses.

B. At the implementation level, you can split a large memory pool lock into multiple small locks, reducing lock scrambling.

2. When the allocated memory pool is depleted and needs to be expanded, apply a chunk of memory at once to increase the efficiency of the expansion. As shown in Level 0.

3. The code implementation Gmmpool,bench results show a performance increase of about 19 times times:

BenchmarkStdReadAll-4 200000 5969 ns/op
BenchmarkMultiLevelPool-4 5000000 311 Ns/op

Summary

For scenarios where memory allocations and releases are frequent, using a memory pool can significantly reduce the overhead of the Golang runtime. Also note that memory pool memory to the user management, you need to carefully check whether there is a memory leak problem. If you are not so harsh on performance requirements, just want to reuse some small objects, then we recommend you use the standard library of sync. Pool.

In addition, the initial mention of Alibaba cloud performance problem, even using the memory pool optimization, the result is very tragic. Finally Alibaba Cloud helped us replace the machine that did not hit the toothpaste factory fix. is not very pleasantly surprised.

Source: https://liudanking.com/arch/golang-multi-level-memory-pool-design-implementation/

Copyright statement: The content source network, the copyright belongs to the original person. Unless it is impossible to confirm, we will mark the author and source, and if there is any infringement please inform us that we shall immediately delete and apologize. Thank you.