Go deep into NodeJS-Buffer

Go deep into NodeJS-Buffer

Javascript is very friendly to string processing. Both wide-byte and single-byte strings are considered as a string. Node needs to process network protocols, operate databases, process images, upload files, and so on. It also needs to process a large amount of binary data. The built-in strings are far from meeting these requirements, so the Buffer came into being.

Buffer Structure

Buffer is a typical module that combines Javascript and C ++. The performance-related module is implemented using C ++, and the non-performance-related module is implemented using javascript.

When the Node starts, the Buffer has been installed into the memory and placed into the global object. Therefore, require is not required.

Buffer object: similar to an array, the element is a hexadecimal two-digit number.

Buffer Memory Allocation

The Buffer object memory allocation is not in the V8 heap memory, and the memory application is implemented at the Node C ++ layer.

To efficiently use the applied memory, Node uses the slab allocation mechanism. slab is a dynamic memory management mechanism that applies various * nix operating systems. Slab has three statuses:

(1) full: full allocation status

(2) partial: partial allocation status

(3) empty: not allocated

Buffer conversion Buffer objects can be converted with strings, supported encoding types: ASCII, UTF-8, UTF-16LE/UCS-2, Base64, Binary, Hex String to BufferNew Buffer (str, [encoding]), default UTF-8buf.write (string, [offset], [length], [encoding]) Buffer to stringBuf. toString ([encoding], [start], [end]) Encoding types not supported by BufferBuffer. isEncoding (encoding) determines whether iconv-lite is supported: Pure JavaScript implementation, lighter, better performance, no need to convert C ++ to javascript iconv: call the libiconv library of C ++ to complete Buffer splicing. Note res. on ('data', function (chunk) {}), where the chunk parameter is a Buffer object. Directly splicing with + will be automatically converted to a string, for wide-byte characters, garbled characters may occur. solution (1) Use the setEncoding () method in the readable stream to transfer data events instead of Buffer objects, it is the encoded string, which uses the StringEncoder module internally. (2) Save the Buffer object to the array, and finally assemble it into a large Buffer to convert the post-encoding into a string. The output Buffer is widely used in file I/O and network I/O, its performance is more important than that of common strings. In addition to the performance loss of String Conversion, the use of Buffer has a highWaterMark setting that is critical to the performance impact during file reading. A. The highWaterMark setting has a certain impact on the allocation and use of Buffer memory. B. The highWaterMark setting is too small, which may lead to too many system calls.