Nodejs Buffer Module Detailed

First, the opening analysis

The so-called buffer buffers, which means "temporary storage area", is a memory that temporarily stores input and output data.

The JS language itself has only a string data type and no binary data type, so Nodejs provides a string-equivalent global constructor buffer to provide operations on binary data. In addition to being able to read a file to get an instance of buffer, it can be constructed directly, for example:

Copy CodeThe code is as follows:
var buffer = new buffer ([0x68, 0x65, 0x6c, 0x6c, 0x6f]);

Buffer is similar to a string, except that you can use the. Length property to get the byte length, and you can use [index] to read bytes at a specified position, for example:

Copy CodeThe code is as follows:
BUFFER[0]; 0x68

Buffer and string can be converted to each other, for example, the binary data can be converted to a string using the specified encoding:

Copy CodeThe code is as follows:
var str = buffer.tostring ("Utf-8"); Hello

Converts a string to the binary data under the specified encoding:

Copy CodeThe code is as follows:
var buffer= new buffer ("Hello", "utf-8"); <buffer 6c 6c 6f>

A little bit different:

There is one important difference between buffer and string. The string is read-only and any modifications to the string are given a new string, and the original string remains unchanged.

As for buffer, it is more like a C-language array that can do pointer manipulation. For example, you can modify the byte of a location directly by using the [index] method.

--------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------- -------

The slice method does not return a new buffer, but rather returns a pointer to a location in the middle of the original buffer, as shown below.

[0x68, 0x65, 0x6c, 0x6c, 0x6f]
^           ^
| |
Bin Bin.slice (2)
Therefore, modifications to the buffer returned by the slice method are used for the original buffer, for example:

Copy CodeThe code is as follows:
var buffer= new Buffer ([0x68, 0x65, 0x6c, 0x6c, 0x6f]);
var sub = Bin.slice (2);
Sub[0] = 0x65;
Console.log (buffer); <buffer 6c 6f>

If you want to copy a buffer, you must first create a new buffer and copy the data from the original buffer through the. Copy method.

This is similar to applying for a new piece of memory and copying the existing in-memory data in the past. Here is an example.

Copy CodeThe code is as follows:
var buffer= new Buffer ([0x68, 0x65, 0x6c, 0x6c, 0x6f]);
var dup = new Buffer (bin.length);
Buffer.copy (DUP);
Dup[0] = 0x48;
Console.log (buffer); <buffer 6c 6c 6f>
Console.log (DUP); <buffer 6c 6f>

In summary, buffer expands the data processing capability of JS from a string to any binary data.

The above simple let us know what is the buffer, the following specific how to use and specific use of the scene.

Two, talk about buffer.

JavaScript is very friendly to string processing, whether it is a wide-byte or single-byte string, and is considered a string. Node needs to deal with network protocols, manipulate databases, process pictures, file uploads, and so on, and also deal with a large number of binary data, the resulting string is far from meeting these requirements, so buffer was born.

Buffer structure

Buffer is a typical JavaScript and C + + module, performance-related parts are implemented in C + +, non-performance-related parts are implemented with JavaScript.

When node starts the process, buffer is loaded into memory and placed into the global object, so there is no need to require

Buffer object: Similar to an array whose elements are two-digit 16 binary digits.

Buffer memory allocation

The memory allocation of the buffer object is not in the heap memory of V8, which implements the memory request at node's C + + level.

In order to efficiently use applications for memory, node uses slab allocation mechanism, slab is a dynamic memory management mechanism, application of various *nix operating systems. There are three states of slab:

(1) Full: Fully allocated status

(2) Partial: Partial distribution status

(3) Empty: not Assigned status

Conversion of buffer

The buffer object can be converted to and from a string, supported by the following types of encodings:

ASCII, UTF-8, Utf-16le/ucs-2, Base64, Binary, Hex

String to Buffer

New Buffer (str, [encoding]), default UTF-8
Buf.write (string, [offset], [length], [encoding])

Buffer to String

Buf.tostring ([encoding], [start], [end])

Type of encoding not supported by buffer

Determine whether the support is supported by buffer.isencoding (encoding)

Iconv-lite: pure JavaScript implementation, lighter, better performance without C + + to JavaScript conversion

Iconv: Call the C + + Libiconv library to complete

The splicing of buffer

Note that "Res.on (' Data ', function (chunk) {})", where the parameter chunk is a buffer object, is automatically converted to a string using the + concatenation, which can result in garbled characters for wide-byte character.

Workaround:

(1) through the Setencoding () method in a readable stream, this method allows the data event to be passed as a buffer object instead of the encoded string, which uses the Stringencoder module internally.

(2) The buffer object is staged into the array and finally assembled into a large buffer to convert the encoding to a string output.

Buffer is widely used in file I/O and network I/O, and its performance is significant, much higher than normal string performance.

The use of buffer in addition to the conversion of the string has a performance loss, when the file is read, there is a highwatermark setting is critical to the performance impact.

The A,highwatermark setting has an impact on the allocation and use of buffer memory.

B, the Highwatermark is too small and may cause too many system calls.

When to use buffer, when should not be used------pure JavaScript support Unicode code and the binary is not very support, when the TCP stream or file flow, processing flow is necessary, we save the non-utf-8 string, 2 binary and so on other formats, we have to use "Buffer".

Third, the introduction of the example

Copy CodeThe code is as follows:
var buf = new Buffer ("This is the text concat test!"), str = "This is the text concat test!";
Console.time ("Buffer concat test!");
var list = [];
var len = 100000 * buf.length;
for (Var i=0;i<100000;i++) {
List.push (BUF);
Len + = Buf.length;
}
var S1 = buffer.concat (list, Len). toString ();
Console.timeend ("Buffer concat test!");
Console.time ("String concat test!");
var list = [];
for (var i = 100000; I >= 0; i--) {
List.push (str);
}
var s2 = list.join ("");
Console.timeend ("String concat test!");

The following is the result of the operation:

The read speed is sure that the string is faster, and buffer requires the operation of ToString (). So when we save the string, we should use a string or string, even if the large string stitching string is not slower than buffer.

So when do we need to use buffer? No way, when we save the non-utf-8 string, 2 binary and other formats, we have to use.

Four, summarize

(1), JavaScript is suitable for processing Unicode encoded data, but the processing of binary data is not friendly.
(2), so it is necessary to handle the TCP stream or file system for the processing of the eight-bit byte stream.
(3), node has several methods for processing, creating and consuming eight-bit byte streams.
(4), the original data is stored in a buffer instance, a buffer similar to an array of integers, but its memory, allocated outside the V8 stack. The size of a buffer cannot be changed.
(5), the type of encoding processed is: ASCII,UTF8,UTF16LE,UCS2 (Utf16le alias), Base64,binary,hex.
(6), buffer is a global element, and a buffer instance is obtained directly from new buffer ().

Nodejs Buffer Module Detailed