Big Bear nodejs------Buffer Module

First, the opening analysis

The so-called buffer buffers, which means "temporary storage area", is a memory that temporarily stores input and output data.

The JS language itself has only a string data type and no binary data type, so Nodejs provides a String global constructor with peers Buffer to provide operations on binary data. In addition to the instances that can be read from Buffer the file, it can be constructed directly, for example:

var buffer = new buffer ([0x68, 0x65, 0x6c, 0x6c, 0x6f]);

BufferSimilar to strings, in addition to the ability to .length get a byte length with a property, you can also [index] read bytes at a specified location, for example:

// 0x68;

Bufferand strings can be converted to each other, for example, the binary data can be converted to a string using the specified encoding:

var str = buffer.tostring ("Utf-8");  // Hello

Converts a string to the binary data under the specified encoding:

var New // <buffer 6c 6c 6f>

A little bit different:

BufferThere is an important difference from a string. The string is read-only and any modifications to the string are given a new string, and the original string remains unchanged.

As Buffer for, it is more like a C-language array that can do pointer manipulation. For example, you can [index] modify the bytes of a location directly by using the method.

--------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------- -------

slicemethod does not return a new one Buffer , it is more like returning a Buffer pointer to a location in the middle of the original, as shown below.

[ 0x68, 0x65, 0x6c, 0x6c, 0x6f ] ^ ^ | | bin bin.slice(2)

The slice modifications returned by the method are therefore used for the Buffer original Buffer , for example:

1 var New Buffer ([0x68, 0x65, 0x6c, 0x6c, 0x6f ]); 2 var sub = Bin.slice (2); 3 sub[0] = 0x65 ; 4 //   <buffer 6c 6f>

If you want to make a copy Buffer , you must first create a new one Buffer and then .copy copy the data from the original by means of the method Buffer .

 1  var  buffer= new  Buffer ([0x68, 0x65, 0x6c, 0x6c, 0x6f 2  var  dup = new   Buffer (bin.length);  3  bin.copy (DUP);  4  dup[0] = 0x48  5  console.log (buffer); //  <buffer 6c 6c 6f>   6  Console.log (DUP); //  <buffer 6c 6f>  

In a word, the Buffer data processing ability of JS is extended from string to any binary data.

The above simple let us know what is the buffer, the following specific how to use and specific use of the scene.

Two, talk about Buffer .

JavaScript is very friendly to string processing, whether it is a wide-byte or single-byte string, and is considered a string. Node needs to deal with network protocols, manipulate databases, process pictures, file uploads, and so on, and also deal with a large number of binary data, the resulting string is far from meeting these requirements, so buffer was born.

Buffer Structure

Buffer is a typical JavaScript and C + + module, performance-related parts are implemented in C + +, non-performance-related parts are implemented with JavaScript.

When node starts the process, buffer is loaded into memory and placed into the global object, so there is no need to require

Buffer object: Similar to an array whose elements are two-digit 16 binary digits.

Buffer memory allocation

The memory allocation of the buffer object is not in the heap memory of V8, which implements the memory request at node's C + + level.

In order to efficiently use applications for memory, node uses slab allocation mechanism, slab is a dynamic memory management mechanism, application of various *nix operating systems. There are three states of slab:

(1) Full: Fully allocated status

(2) Partial: Partial distribution status

(3) Empty: not Assigned status

Conversion of buffer

The buffer object can be converted to and from a string, supported by the following types of encodings:

ASCII, UTF-8, Utf-16le/ucs-2, Base64, Binary, Hex

String to Buffer

New Buffer (str, [encoding]), default UTF-8

Buf.write (string, [offset], [length], [encoding])

Buffer to String

Buf.tostring ([encoding], [start], [end])

Type of encoding not supported by buffer

Determine whether the support is supported by buffer.isencoding (encoding)

Iconv-lite: pure JavaScript implementation, lighter, better performance without C + + to JavaScript conversion

Iconv: Call the C + + Libiconv library to complete

The splicing of buffer

Note that "Res.on (' Data ', function (chunk) {})", where the parameter chunk is a buffer object, is automatically converted to a string using the + concatenation, which can result in garbled characters for wide-byte character.

Workaround:

(1) through the Setencoding () method in a readable stream, this method allows the data event to be passed as a buffer object instead of the encoded string, which uses the Stringencoder module internally.

(2) The buffer object is staged into the array and finally assembled into a large buffer to convert the encoding to a string output.

Buffer is widely used in file I/O and network I/O, and its performance is significant, much higher than normal string performance.

The use of buffer in addition to the conversion of the string has a performance loss, when the file is read, there is a highwatermark setting is critical to the performance impact.

The A,highwatermark setting has an impact on the allocation and use of buffer memory.

B, the Highwatermark is too small and may cause too many system calls.

When to use buffer, when should not use------ Pure JavaScript supports Unicode code and is not very supportive of binary, when resolving TCP stream or file stream, processing flow is necessary, we save non-utf-8 string, 2 binary and other formats, we must use " Buffer " .

Third, the introduction of the example

1 varBUF =NewBuffer ("This is the text concat test!"), str = "This is Text concat test!" ;2Console.time ("Buffer concat test!"));3 varList = [] ;4 varLen = 100000 *buf.length;5  for(vari=0;i<100000;i++){6 List.push (BUF);7Len + =buf.length;8 }9 varS1 =buffer.concat (list, Len). toString ();TenConsole.timeend ("Buffer concat test!")) ; OneConsole.time ("String concat test!")) ; A varList = [] ; -  for(vari = 100000; I >= 0; i--) { - List.push (str); the } - varS2 = List.join ("") ; -Console.timeend ("String concat test!");

The following is the result of the operation:

The read speed is sure that the string is faster, and buffer requires the operation of ToString (). So when we save the string, we should use a string or string, even if the large string stitching string is not slower than buffer.

So when do we need to use buffer? No way, when we save the non-utf-8 string, 2 binary and other formats, we have to use.

Four, summarize

(1), JavaScript is suitable for processing Unicode encoded data, but the processing of binary data is not friendly.

(2), so it is necessary to handle the TCP stream or file system for the processing of the eight-bit byte stream.

(3), node has several methods for processing, creating and consuming eight-bit byte streams.

(4), the original data is stored in a buffer instance, a buffer similar to an array of integers, but its memory, allocated outside the V8 stack. The size of a buffer cannot be changed.
(5), the type of encoding processed is: ASCII,UTF8,UTF16LE,UCS2 (Utf16le alias), Base64,binary,hex.
(6),buffer is a global element, and a buffer instance is obtained directly from new buffer ().

　　　　　　　　　　　　　　　　　　　　　　　　　　　　hahaha, the end of this article, not to be continued, hope and we have enough communication, common progress ... To shout ... (*^__^*)

Big Bear nodejs------Buffer Module