Readable and writable streams in node

One disadvantage of JavaScript is that binary data cannot be processed, so the buffer type is introduced in node. This type allocates storage space for the data in one byte (that is, 8 bits). It is used like an array, but differs from array: buffer must know its length clearly when defined, but the length of the array can be changed dynamically. There are three ways of defining buffer:

1. var buf = new buffer (3);//Specify buffer to occupy 3 bytes

2. var buf = new Buffer ("Hello", "utf-8");//Specifies the string, and the encoding type of the string is Utf-8, which occupies 5 bytes

3. var buf = new Buffer ([256,255,2]),//defines the number stored for each byte of buffer, maximum 255, more than 255 modulo

Why did you introduce buffer? Because the flow described in this article is transmitted in bytes, the buffer stores the bytes.

Node uses the Createreadstream and Createwritestream of the FS module to create readable and writable streams, respectively. The first interface is described first.

I. Createreadstream

This method inherits the Eventemitter class, so it can also emit and listen for related events. Its usage is as follows:

var rs = Fs.createreadstream (FilePath, {options});
Rs.on ("Data", function (data) {
Console.log (data);
});
Rs.on ("End", function (data) {
Console.log ("Resource file is completely read");
});
Rs.on ("Error", function (Err) {
Console.log ("Something is wrong during processing");
})

Options is a set of key-value values that are commonly used as follows:

Flags: What to do with the file, default to ' R ', read file

Encoding: Specifies the encoding, default is NULL, if no specific encoding format is set, the data read is the buffer type, or you can use Rs.setencoding ("Utf-8") to specify the encoding format

Start: Read file from start

End: Reads the file to end (including end)

Highwatermark: The maximum watermark, the internal buffer can hold the maximum number of bytes, if more than this size, stop reading the resource file, the default value is 64KB

The first few settings follow the literal meaning and are better understood. Focus on Highwatermark.

For example, there is a 100KB file, set Highwatermark to 10KB, then the system will first read the data from the resource file, and then trigger the data event, and then read the 10KB, trigger the database event, continue to execute, until all the data is read out, Triggers the end event. Highwatermark can not be set too small, too small will be frequent operation of files, affecting performance.

There are two important ways to read a stream: Pause and resume, respectively, to suppress the launch of the data event and to activate the transmit data event. These two methods are explained later.

Two. Createwritestream

Like Createreadstream, it is also a subclass of Eventemitter. Its drain event is triggered after the buffer data has been written.

Examples of its usage:

var ws = Fs.createwritestream (FilePath, {options});

The common options values are:

Flags: What to do with the file, the default is "W", representing the write file, if "a", representing each write, not empty the original data in the file, but directly at the end of the original data to add new data

Encoding: Specifies the encoding format to write, default to NULL

Start: Where the file begins to write

Highwatermark: The maximum number of bytes the buffer can hold, default is 16KB, and if this value is exceeded, the Write method returns false

The Highwatermark of the writable stream also represents the capacity of the buffer, and if the buffer is already full, continuing to write the data will fail. The Write method can be recalled to write to the next Highwatermark size data chunk only after the contents of the buffer are written to the file.

There are two important ways to write a writable stream: Write and End,write define what is written, and end can forcibly write to the file and close the destination file (it cannot continue writing).

Based on the above analysis, we will implement an example of a file copy:

varFs=require ("FS");varrs = Fs.createreadstream ("./koala.jpg");//default 64KBvarWS = Fs.createwritestream ("./copy.jpg");//default 16KB, write speed is less than read speedRs.on ("Data",function(data) {varFlag =ws.write (data); if(!flag) {//buffer is fullRs.pause ();//stop triggering the data event}}); Ws.on ("Drain",function() {rs.resume ();//The data event is re-triggered if the current buffer is finished writing}); Rs.on ("End",function() {ws.end ();//Write all the remaining data, andto close a file that is written})

This example combines the events and important methods of readable and writable streams. Since the read stream is faster than writing the inflow, control the flow of data in the readable stream. This problem is solved by controlling the timing of the data event's triggering.

Readable and writable streams in node