Introduction to node Buffer

Well, I thought the Buffer in node is only a small part of node. js. After carefully reading some information, I found that the glacier always hides its huge ass below sea level, this is also a simple introduction to some of the shortest Buffer (for beginners of Node ~)

What is Buffer?
For JavaScript, Unicode-encoded data is easy to process, but there is no way to process binary data. However, when some TCP data streams or file data streams are operated, the byte stream processing method is required. In nodeJS, a processing method is provided ~ Provides a global constructor Buffer that is similar to a String (it is better to say that it is similar to an Array, but you still need to pay attention to some different aspects ), in this case, do not use require every time. Buffer is the intermediary in the node that stores binary data.

How to create a Buffer instance: new Buffer (size );
Similar to an array, the size of the Buffer object to be created is given.



New Buffer (array );
An array of binary data.



New Buffer (str, [encoding]);
Data of a specific data type and Its Encoding type (utf8 by default)




There are also some methods to operate the Buffer instance:
A Buffer is very similar to an integer array, which is different from a string.
Javascript programmers are familiar with it. For strings, they can be considered as readable. Each modification and copying will get a new string without affecting the previous string. For example, there is a string, var B = a; a and B each have an independent string data, which is different from the object in a pointer format (pass by value and by address ).
However, for the sibling Buffer of the string, the Buffer of the object contains more Array blood.



However, when using some Buffer methods, you must note that they are inconsistent with the array.




After intercepting a part of the created Buffer, modifying the returned Buffer segment will still affect the original Buffer, and the two are still associated. This is similar to an array that stores objects. Each element in the Buffer array is a reference value instead of an original value.

Buffer provides a special method for copying an independent Buffer copy.
Buffer. copy (targetBuffer, [targetStart], [sourceStart], [sourceEnd]);
Relatively speaking, there is no direct copy method. You can only create a new Buffer with the same length and then call the copy method on the original Buffer, you can also set the notice and end position of copy.




For more information, see the Buffer document http://nodejs.org/api/buffer.html for node

When does Buffer need to be used?
Well, the above is a brief introduction to the Buffer. In fact, it is all virtual. When do we need to use the Buffer?
In fact, we have already said that in Node, data streams in many places are of the Buffer type used for processing binary data (such as reading file data, data transmitted by post in an http request ).

 
 

  
  
var fs = require('fs');

  
  
var rs = fs.createReadStream('chinese.md');

  
  


  
  
rs.on("data", function (chunk){

  
  
console.log( Buffer.isBuffer( chunk ) )

  
  
});
 
 

We can see from the above that all of them are Buffer fragments, so in some cases we will also encounter some hidden problems:

 
 

  
  
var fs = require('fs');

  
  
var rs = fs.createReadStream('chinese.md' , { highWaterMark: 5 } );

  
  
var data = '';

  
  
rs.on("data", function (chunk){

  
  
data += chunk;

  
  
});

  
  
rs.on("end", function () {

  
  
console.log(data);

  
  
});
 
 

Garbled characters may occur and cannot be displayed normally.



A text occupies three Buffer units, but when the data stream is segmented, the three identifiers of a Chinese character are separated. This garbled code occurs when the toString is created. The solution is that the block data obtained must not be directly escaped using the toString type. Each Buffer segment is saved, merged into a large Buffer, and escaped:

 
 

  
  
var fs = require('fs');

  
  
var rs = fs.createReadStream('chinese.md', { highWaterMark: 5 } );

  
  
var dataArr = [], len = 0, data;

  
  
rs.on("data", function (chunk){

  
  
dataArr.push(chunk);

  
  
len += chunk.length;

  
  
});

  
  
rs.on("end", function () {

  
  
data = Buffer.concat( dataArr, len ).toString();

  
  
console.log(data);

  
  
});
 
 

In the above example, we may also find a problem. After removing the {highWaterMark: 5} parameter, there is actually no problem between the two because the default unit of data block cut in stream is 8 K, therefore, this small instance can be stored in the unit content, so there is no problem. Once it is said that the data is large, there will also be problems. (I have been using txt for a long time before, and found that it is not correct. I found that it is a problem with the encoding method ).

By the way, in terms of splicing performance, Buffer is not very bad compared with String:

 
 

  
  
var buf = new Buffer('this is text concat test'),

  
  
str = 'this is text concat test';

  
  
console.time('buffer concat test');

  
  
var list = [];

  
  
var len= 100000 * buf.length;

  
  
for(var i=0;i<100000;i++){

  
  
list.push(buf);

  
  
len += buf.length;

  
  
}

  
  
var s1 = Buffer.concat(list, len).toString();

  
  
console.timeEnd('buffer concat test');

  
  
console.time('string concat test');

  
  
var list = [];

  
  
for (var i = 100000; i >= 0; i--) {

  
  
list.push(str);

  
  
}

  
  
var s2 = list.join('');

  
  
console.timeEnd('string concat test');
 
 

Okay, I admit that I am a little impatient with writing. That's it. It's like a simple tag. You need to check it again ~ Oh, roar ~