The node.js of the flow writing tool in the _node.js is a matter to be noted

The flow in Node.js is powerful, it provides support for handling potentially large files, and also abstracts data processing and delivery in some scenarios. Because it is so useful, so in the actual combat we often based on it to write some tool functions/libraries, but often due to their own convective characteristics of the negligence, resulting in the function/library in some cases will not achieve the desired effect, or buried some hidden mines. This article will provide two of the two tips that are used in writing streaming based tools.

One, beware of eventemitter memory leaks

In a function that may be called multiple times, if you need to add an event listener to the stream to perform some action. Then you need to be wary of memory leaks caused by adding listeners:

' Use strict ';
Const FS = require (' FS ');
Const CO = require (' co ');

function Getsomedatafromstream (stream) {let
 data = Stream.read ();
 if (data) return promise.resolve (data);

 if (!stream.readable) return promise.resolve (null);

 return new Promise (Resolve, Reject) => {
  stream.once (' readable ', () => Resolve (Stream.read ()));
  Stream.on (' Error ', reject);
  Stream.on (' end ', resolve);
 }
}

Let stream = Fs.createreadstream ('/path/to/a/big/file ');

Co (function * () {let
 chunk;
 while ((chunk = yield getsomedatafromstream (stream))!== null) {
  console.log (chunk);
 }
}). catch (Console.error);

In the above code, the Getsomedatafromstream function completes this promise by listening for error events and end events to stream errors or without data. However, when executing the code, we will soon see the alert message in the console: (node) warning:possible Eventemitter memory leak detected. One error listeners added. Use Emitter.setmaxlisteners () to increase limit. Because each time we call the function, we add an additional error event listener and an end event listener for the incoming stream. To avoid this potential memory leak, we want to make sure that every time the function finishes, clear all additional listeners added by this call and keep the function clean:

function Getsomedatafromstream (stream) {let
 data = Stream.read ();
 if (data) return promise.resolve (data);

 if (!stream.readable) return promise.resolve (null);

 return new Promise (Resolve, Reject) => {
  stream.once (' readable ', ondata);
  Stream.on (' Error ', onError);
  Stream.on ("End", done);

  function OnData () {done
   ();
   Resolve (Stream.read ());
  }

  function OnError (err) {done
   ();
   Reject (err);
  }

  function done () {
   stream.removelistener (' readable ', ondata);
   Stream.removelistener (' Error ', onError);
   Stream.removelistener (' End ', done);}}}

Second, ensure that the callback of the tool function is not invoked until the data is processed.

Tool functions often provide a callback function argument externally, after all the data in the stream is to be processed, triggering with the specified value is usually done by hanging the call to the callback function in the end event of the stream, but if the handler is a time-consuming asynchronous operation, the callback function may be invoked before all data is processed:

' Use strict ';
Const FS = require (' FS ');

Let stream = Fs.createreadstream ('/path/to/a/big/file ');

function Processsomedata (stream, callback) {
 stream.on (' data ') => {
  ///To perform some asynchronous time-consuming operations on the data
  settimeout (() => console.log (data);
 });

 Stream.on (' End ', () => {
  //...
  Callback ()
 })
}

Processsomedata (Stream, () => console.log (' End '));

The above code callback callback may be invoked when the data is not fully processed, because the end event of the stream is triggered only when the data in the stream is read. So we need to check that the data has been processed in extra detail:

function Processsomedata (stream, callback) {let
 count = 0;
 Let finished = 0;
 Let isend = false;

 Stream.on (' Data ', (data) => {
  count++;
  Perform some asynchronous time-consuming operations on the data
  settimeout (() => {
   console.log (data);
   finished++;
   Check ();
  },%);
 };

 Stream.on (' End ', () => {
  isend = true;
  // ...
  Check ();
 })

 function Check () {
  if (count = = finished && isend) callback ()
 }
}

As a result, the callback is triggered after all data has been processed.