Node. js advanced programming: using Javascript to build scalable applications (4) 2.4 core API basics-using Buffer for processing, encoding, and decoding of binary data

Document directory

• Create a buffer
• Obtain and set buffered data
• Split and buffer data
• Copy buffered data
• Decodes buffered data

For the list of articles in this series and the translation progress, see Node. js advanced programming: using Javascript to build scalable applications (〇)

This article corresponds to the original article Part 2 Chapter 4: Node Core API Basics: Using Buffers to Manipulate, Encode, and Decode Binary Data

Copy the article from Word here, especially the indentation is not the same as the original text. You can click here to download the PDF version of this article.

Chapter 4: Buffer processing, encoding, and decoding of binary data:

• Why buffer?
• Create a buffer with a string
• Converts a buffer to a string.
• Process buffered data
• Sharding and replication of buffered data

 

JavaScript is good at processing strings, but it is not very good at processing binary data because it was originally designed to process HTML documents. JavaScript has no byte type, no structured type (structured types), or even no byte array, only numbers and strings. (Original article: JavaScript doesn' t haveByteType-it just has numbers-or structured types, or even byte arrays: It just has strings .)

Because Node is based on JavaScript, it can naturally process text protocols like HTTP, but you can also use it to interact with databases, process images or file uploads, etc. As you can imagine, how difficult it is to use strings to do these tasks. Earlier in the year, Node encoded byte into text characters to process binary data. However, this method was proved to be not feasible, which wastes resources, is slow, and not flexible, and is difficult to maintain.

Node has a binary Buffer to implement Buffer. This pseudo class provides a series of APIS for processing binary data, which simplifies the tasks for processing binary data. The buffer length is determined by the length of the byte data, and you can randomly set and obtain the byte data in the buffer.

Note: BufferClass has a special place, the memory occupied by the byte data in the buffer is not allocated in JavaScrp

It VMMemory on the heap, that is, these objects will not be JavaScriptInstead of a permanent memory address that will not be modified, this avoids the CPU caused by memory replication of buffered content.Waste.

Create a buffer

You can create a buffer with a UTF-8 string like this:

Var buf = new Buffer ('Hello World! ');

You can also use a specified encoded string to create a buffer:

Var buf = new Buffer ('8b76fde713ce ', 'base64 ');

Acceptable character codes and identifiers are as follows:

• Ascii-- ASCI, only applicable to ASCII character sets.
• Utf8--utf-8: this variable-width encoding applies to any character set in the Unicode character set. It has become the preferred encoding in the Web world and is also the default encoding type of Node.
• Base64-- Base64: This encoding is based on 64 printable ASCII characters to represent binary data. Base64 is usually used to embed binary data that can be converted into strings in character documents, when necessary, it can be completely and lossless converted back to the original binary format.

If there is no data to initialize the buffer, you can use the specified capacity to create an empty Buffer:

Var buf = new Buffer (1024 );//Create a 1024Byte Buffer

Obtain and set buffered data

After creating or receiving a buffer object, you may want to view or modify its content. You can use the [] operator to access a buffer byte:

Var buf = new Buffer ('My buffer content ');

//Access buffer 10thBytes

Console. log (buf [10]); //-> 99

Note: When you (using the buffer capacity) Create an initialized buffer, note that the buffered data is not initialized to 0.But random data.

Var buf = new Buffer (1024 );

Console. log (buf [100]); //-> 5 (a random value)

You can modify the data at any location in the buffer as follows:

Buf [99] = 125 ;//Set 100thThe value of each byte is set to 125.

Note: In some cases, some buffer operations do not produce errors, such:

• The maximum number of bytes in the buffer is 255.If a byte is given a value greater than 256Will use 256Modulo it, And then assign the result to this byte.
• If you assign a value of 256 to a buffer byteThe actual value is 0.(Note: it is actually the same as the first article, 256% 256 = 0)
• If a floating point value is used to assign a value to a byte in the buffer, such as 100.7, The actual value will be the integer part of the floating point -- 100
• If you try to assign a value to a location that exceeds the buffer capacity, the value assignment operation will fail and the buffer will not be modified.

You can use the length attribute to obtain the buffer length:

Var buf = new Buffer (100 );

Console. log (buf. length); //-> 100

You can also use the buffer length to iterate the buffer content to read or set each byte:

Var buf = new Buffer (100 );

For (var I = 0; I <buf. length; I ++ ){

Buf [I] = I;

}

The above Code creates a buffer containing 100 bytes and sets each byte in the buffer from 0 to 99.

Split and buffer data

Once a buffer is created or received, you may need to extract a portion of the buffered data. You can specify the starting position to split the existing buffer and create another small buffer:

Var buffer = new Buffer ("this is the content of my buffer ");

Var smallerBuffer = buffer. slice (8, 19 );

Console. log (smallerBuffer. toString (); //-> "the content"

Note: When splitting a buffer, no new memory is allocated or copied. The new buffer uses the parent buffer memory, which is only used by the parent buffer to buffer certain data segments (specified by the starting position). This section contains several meanings.

First, if your program modifies the content of the parent buffer, these modifications will also affect the related sub-buffer, because the parent buffer and sub-buffer are different JavaScript objects, therefore, it is easy to ignore this issue and cause some potential bugs.

Second, when you use this method to create a small sub-buffer from the parent buffer, the parent buffer object will still be retained after the operation is completed and will not be reclaimed. If you do not pay attention to it, it is easy to cause memory leakage.

Note: If you are worried about memory leakage, you can use copyMethod to replace sliceOperation. Next we will introduce the copy.

Copy buffered data

You can use copy to copy a part of the buffer to another buffer as follows:

Var buffer1 = new Buffer ("this is the content of my buffer ");

Var buffer2 = new Buffer (11 );

Var targetStart = 0;

Var sourceStart = 8;

Var sourceEnd = 19;

Buffer1.copy (buffer2, targetStart, sourceStart, sourceEnd );

Console. log (buffer2.toString (); //-> "the content"

The code above copies the 9th to 20 bytes of the source buffer to the starting position of the target buffer.

Decodes buffered data

Buffer data can be converted to a UTF-8 string like this:

Var str = buf. toString ();

You can also specify the encoding type to decode the buffered data into any encoding type. For example, if you want to decode a buffer into a base64 string, you can do this:

Var b64Str = buf. toString ("base64 ");

Using the toString function, you can also transcode A UTF-8 string to a base64 string:

Var utf8String = 'my string ';

Var buf = new Buffer (utf8String );

Var base64String = buf. toString ('base64 ')

Summary

Sometimes, you have to deal with binary data, but native JavaScript does not have a clear way to do this, so Node provides a Buffer class that encapsulates some operations for contiguous memory blocks. You can split or copy memory data between two buffers.

You can also convert a buffer into a encoded string or, in turn, convert a string into a buffer to access or process each bit.