Analysis on the buffer class of Nodejs

1, when to use the buffer, when should not be used to look at the following test code, respectively, splicing a variety of different length of the string, and finally directly splicing 10MB string

var String,string2,string3;var Bufstr,bufstr2,bufstr3;var J; Console.time (' Write + string '); for (j=0;j<1000;j++) {var x = j+ "; string + = x;} Console.timeend (' Write + string '); Console.time (' Write buffer '); bufstr = new buffer (+); for (j=0;j<1000;j+ +) {var x = j+ '; bufstr.write (x,j);} Console.timeend (' Write buffer '); Console.time (' Write 100000 string '); for (j=0;j<100000;j++) {var x = j+ '; string2 + = x;} Console.timeend (' Write 100000 string '); Console.time (' Write 100000 buffer '); bufstr2 = new buffer (100000) for (j=0;j< 100000;j++) {var x = j+ '; bufstr2.write (x,j);} Console.timeend (' Write 100000 buffer '); Console.time (' Write 1024*1024*10 string '); for (j=0;j<1024*1024*10;j++) { var x = j+ "; string3 + = x;} Console.timeend (' Write 1024*1024*10 string '); Console.time (' Write 1024*1024*10 buffer '); BUFSTR3 = new Buffer (1024*1024 *10); for (j=0;j<1024*1024*10;j++) {var x = j+ "; bufstr3.write (X,J);} Console.timeend (' Write 1024*1024*10 buffer '); 
 

　　

Then the output:

The read speed does not need to be tested, the string is sure to be faster, and buffer requires the operation of ToString (). So when we save the string, we should use a string or string, even if the large string stitching string is not slower than buffer. So when do we need to use buffer? No way, when we save the non-utf-8 string, 2 binary and other formats, we have to use.

2, Buffer had to mention the 8KB

Buffer famous 8KB carrier, for example, node put a big house into a lot of small rooms, each room can accommodate 8 people, in order to ensure the full use of the room, only when a room stuffed with 8 people will open a new room, but when there are multiple people to stay, Node will be guaranteed to put these people in a room, such as the current room A has 4 people live, but suddenly came 5 people, so node had to open a new room B, the 5 people settled down, then came 4 people, found that 5 people's B room also can not accommodate, only open a room C, So everyone is settled down. But the previous two rooms A and b each wasted 4 and 3 positions, and room C became the current room.

Specifically, when we instantiate a new buffer class, according to the size of the instantiation to apply for memory space, if the required space is less than 8KB, it will be more time to determine whether the current 8KB carrier remaining capacity is enough new buffer instance, if enough, The new buffer instance is saved in the current 8KB vector and the remaining space is updated.

Let's do a simple experiment to simulate a more serious memory leak scenario:

The first time we put the memory leak point that line of code commented out, after 4 minutes of running, to get the following printing information, V8 has automatically freed my allocated memory, free men back to the beginning of the value, the second time we will leak point that line of code release, let the global variable Leak_buf_ary Always refer to buffer and do the same for 10 minutes

VarOs= Require(' OS ');VarLeak_buf_ary= [];VarShow_memory_usage= function(){ Print System free MemoryConsole.Log(' Free mem: ' + Math.Ceil(Os.Freemem()/(1024*1024)) + ' MB ');}VarDo_buf_leak= function(){VarLeak_char= L; Leak of several byte charactersVarLoop= 100000;100,000 playsVarBuf1_ary= []While(Loop--){Buf1_ary.Push(New Buffer(4096)); Apply for BUF1, occupy 4096byte space, will be automatically releasedRequest Buf2, take up a few bytes of space, save its reference to external data, not automatically released//*******Leak_buf_ary.Push(New Buffer(Loop+Leak_char));//*******}Console.Log("Before GC")Show_memory_usage();Buf1_ary= Null;Return;}console. ( "process Start" ) show_memory_usage< Span class= "pun" > () do_buf_leak (); var J =10000; Setinterval (function () {console log ( "after GC" show_ Memory_usage () },1000 *60)             

Results for the first time:

process startfree mem : 5362mbbefore gcfree mem : 5141mbafter gcfree mem : 5163mbafter gcfree mem : 5151mbafter gcfree mem : 5148mbafter gcfree mem : 5556mb

Second result:

Process Startfree Mem: 5692mbBefore Gcfree Mem: 4882mbAfter Gcfree Mem: 4848mbAfter Gcfree Mem: 4842mbafter gcfree mem : 4843mb after gcfree mem : 4816mb after gcfree mem : 4822mb after gcfree mem : 4816mb after gcfree mem : 4809mb after gcfree mem : 4810mb after gcfree mem : 4831mb after gcfree mem : 4830MB     

Although we released the buffer of 4096byte, but because the bytes of the byte is not released, will cause the entire 8KB of memory can not be released, if we continue to execute the loop eventually our system memory will be exhausted, the program will crash. Also, because we are sequentially allocating 4096+ a few bytes of memory, so every 8KB of memory space will be wasted 409Xbyte, after the execution of the loop, we obviously found that the second memory consumption is much larger than the first time. Here we have more than 300MB of memory consumption.

3. Buffer string Connection when we accept the post data, node is sent in the form of a stream, triggering the OnData event, so we see a lot of code that reads:

Varhttp= Require(' HTTP ');http.Createserver(function (Req,Res) { VarBody= ‘‘;Req.On(' Data ',function(Chunk){Console.log (Buffer.isbuffer (chunk))Body+=Chunk})Req.On(' End ',function(){Console.Log(Body) Res. (200, { ' content-type ' :  ' Text/plain '  Res. ' Hello world\n '  }). listen (8124. Log ( ' Server running at http://127.0.0.1:8124/'    

Below we compare the performance difference between the two, the test code:

VarBuf= New Buffer(' nodejsv0.10.4&nodejsv0.10.4&nodejsv0.10.4&nodejsv0.10.4& ');Console.Time(' String + = Buf ')VarS= ‘‘;For(VarI=0;I<10000;I++){S+=Buf;}S;Console.Timeend(' String + = Buf ')Console.Time(' Buf concat ')VarList= [];VarLen=0;For(VarI=0;I<10000;I++){list.buf+= Buf.; }var S2 =  buffer. Concatlist, Lentostring (); . Timeend ( ' buf concat ' )       

The result of the output, the difference is nearly one times:

string += buf: 15msbuf concat: 8ms

In the 1000 splicing process, the performance of the two is almost one-fold, and when the customer uploads a non-UTF8 string, the direct + = is also prone to errors.

4, exclusive space if you want to create a unique space, independent of the memory space to read and write, there are two ways, 1 is to instantiate a buffer of more than 8KB length, and the other is to use the Slowbuffer class.

5, Buffer Release Unfortunately, we can not manually GC buffer instances, only rely on V8 to do, all we can do is to de-reference the buffer instance.

6. Erase buffer Erase the data on a buffer the fastest way is Buffer.fill

Analysis on the buffer class of Nodejs