Nodejs Read-Write stream

Nodejs Read-Write stream

The transfer process of a stream is transmitted by default in the form of buffer, unless you set him up with another encoded form,
For example, in the second part of the code below, we set the stream to be transmitted in UTF8 encoded form:

1 varReadable stream1 = ...2Readable stream1.on (' data ',function(data) {3 //data is a buffer;4Console.log (' Got this data: ', data);5 });6 varReadable stream2 = ...7Readable stream2.setencoding (' UTF8 '));8Readable stream2.on (' data ',function(data) {9 //data is a utf8-encoded string;TenConsole.log (' Got this data: ', data); One});

The transfer process of the stream can be stopped, and the pause () method can be used to suspend the flow of the stream,
Similarly, the flow of the transmission process has a corresponding recovery method, that is, re-restore the transfer process, using the Resume () method.

Stream.pause ();
Stream.resume ();

We can listen to his end event if we want something to be done after the stream has been lost.

1 var readable stream = ... 2 function () {3     console.log (' The stream has ended '); 4 });

1. Using a writable stream (writable STREAMS)
so-called writable stream, you can have an object that allows you to write data.
The simplest way to manipulate a writable stream is to use the Write method:
var writable_stream = ...;
Writable_stream.write (' This was an UTF-8 string ');
you pass in a string as a parameter, the default is UTF8 encoding, of course, you can also pass in the second parameter in the encoding you want:
var writable_stream = ...;
Writable_stream.write (' 7e3e4acde5ad240a8ef5e731e644fbd1 ', ' base64 ');
Alternatively, you can also write buffer to the writable stream:
var writable_stream = ...;
var buffer = new Buffer (' this was a buffer with some string ');
Writable_stream.write (buffer); One thing to note about
here is that, in the process of writing into a stream, node does not necessarily have the ability to write data to kernel buffer in a timely manner, when node will
store the data in the buffer queue and then re-write the data to the correct location at the appropriate time. At this point it may be necessary to listen for some events to know when the
data for those buffer queues is being re-written, the event is drain.

1 var writable stream = ...; 2 function () {3     console.log (' drain emitted '); 4 });

2. Create a file system stream

var fs = require (' FS ');
var rs = fs.createreadstream ('/path/to/file ');
The above code creates a readable stream, we can pass in the second parameter, the second parameter is a parameter in the form of a JSON object, and has the following configuration items:

1 {2     encoding: ' xx ',3     // file Description 4     BufferSize: ' xx ',5     start: ' xx ',6     end: ' xx '7 }

If you have opened a file that you can read to his file description, you can pass the file description in the second parameter:

1 ar fs = require (' FS '); 2 var path = '/path/to/my/file '; 3 function (Err, FD) {4     fs.createreadstream (null, {FD:FD, encoding: ' UTF8 '}); 5     Fs.on (' Data ', console.log); 6 });

3. Create a file writable stream

var fs = require (' FS ');
var rs = fs.createwritestream ('/path/to/file ', options);
The second parameter defaults to the following values:

1 {flags: ' W ',2null,3 mode:0666}

You can create a writable stream of files that can accept UTF8 encoding:
var fs = require (' FS ');
var rs = fs.createwritestream ('/path/to/file ', {encoding: ' UTF8 '});

4. Understanding Network Flow
A TCP connection is both a read and a writable stream, while an HTTP connection is different, an HTTP request object is a readable stream, and an HTTP response object is a writable stream.
5. Understanding Slow Client issues
First, the server reads files quickly, but the speed of client writes is not consistent with the speed at which the server reads, which causes the server
The cache policy is taken when the data is read without being delivered to the client in a timely manner. This can lead to a serious problem with server memory bursting.
In order to solve this problem, we need to listen to the client writes the event is normal, if can write normally, the server side continues to read the data and transmits.
As mentioned above, the transfer process of the stream can be interrupted and continued, and there are drain events that can be monitored, and we can use these features to optimize our
The process of reading and writing streams. In general, if we do not deal with this problem, the code like the following is very prone to problems:

1Require (' http '). Createserver (function(req, res) {2     varrs = Fs.createreadstream ('/path/to/big/file ');3Rs.on (' Data ',function(data) {4 res.write (data);5     });6Rs.on (' End ',function() {7 res.end ();8     });9}). Listen (8080);

There is no interruption and continuous processing in the code, and the Write () method can return true if the process of writing to the stream is normal, otherwise it will return false;
We can use this API to optimize our code:

1Require (' http '). Createserver (function(req, res) {2     varrs = Fs.createreadstream ('/path/to/big/file ');3Rs.on (' Data ',function(data) {4         if(!res.write (data)) {5 rs.pause ();6         }7     });8Res.on (' drain ',function() {9 Rs.resume ();Ten     }); OneRs.on (' End ',function() { A res.end (); -     }); -}). Listen (8080);

The above procedure is very common when reading and writing streams, so node gives a common method, the pipe (), which solves the above mentioned problems.

1 require (' http '). Createserver (function(req, res) {2     var rs = Fs.createreadstream ('/path/to/big/file '); 3     Rs.pipe (res); 4 }). Listen (8080);

By default, when data transfer is complete, the end () method is called, and if you want to invoke the custom end, you can add a parameter to him:

 1  require (' http '). Createserver (function   (req, res) { 2  var  rs = fs.createreadstream ('/path/to/big/file ' );  3  rs.pipe (res, {end: false   }); 4  rs.on (' End ', function   () { 5  Res.write ("and that's all, folks!"  6   Res.end ();  7  });  8 }). Listen (8080);