Stream in [node. js] node. js

Original address: http://www.moye.me/2015/03/29/streaming_in_node/

What is a stream?

When it comes to flow, it involves a concept of *nix: pipelines-in *nix, streams are implemented in the shell to be able to pass | (pipe character) to bridge the data, the output of a process (stdout) can be directly as input to the next process (stdin).

In node, the concept of stream is similar and represents the ability of a data stream to be bridged.

Pipe

The essence of fluidization is the. Pipe () method. The ability to bridge is bridged by a. Pipe () method on both ends of the data stream (upstream/downstream or called read/write streams).

The pseudo-code behaves as:

Upstream. Pipe (downstream) readable.pipe (writable);

Classification of streams

There is no intention to discuss the "classic" stream before the so-called node v0.4. So, the flow is divided into a few classes (all abstract interfaces:

• Stream. Readable readable stream (requires the implementation of the _read method, the focus is on the details of reading the data stream
• Stream. Writable writable stream (requires implementation of the _write method, the focus is on the details of the data stream writing
• Stream. Duplex read/write stream (need to implement the above two interfaces, the attention point is the details of the above two interfaces
• Stream. Transform inherits from Duplex (_transform method needs to be implemented, the focus is on the processing of data blocks

In simple terms:

• The owner of the pipe () must have the readable stream (not limited to) the ability to have the ' readable '/' data '/' End '/' close '/' ERROR ' series of events available for subscription, also available. Read ()/.pause ()/. A series of methods such as resume () for invocation;
• The parameters of pipe () must have a writable flow (not limited to) capability, it has the ' drain '/' pipe '/' unpipe '/' ERROR '/' Finish ' event to be accessed and also provides a series of methods such as write ()/.end () for invocation

What the heck

Is there any trace of anxiety? Don't worry, as a low-level code worker, I will break the stream and you pull a tear.

The stream class, in the source code of node. js, is defined as:

var EE = require (' Events '). Eventemitter;var util = require (' util '); Util.inherits (stream, EE); function Stream () {  ee.call (this);}

As you can see, stream is essentially a eventemitter, which means it has event-driven functionality (. Emit/.on ... ）。 As we all know, "node. JS is a V8-based event-driven platform" that implements event-driven streaming programming with the characteristics of Node-like asynchronous callbacks.

For example, in a readable stream, there is a readable event that, in a paused read-only stream, is sent to subscribers (readable stream) as long as the data block is ready to be read. Express req,ftp or Mutli-form upload component Req.part, standard input Process.stdin in the system, etc.). With the readable event, we can do a tool such as a parser that handles the output of the shell command:

Process.stdin.on (' readable ', function () {   var buf = Process.stdin.read ();   if (buf) {      var data = buf.tostring ();      Parsing data ...                                                    });

This is called:

Head-10 Some.txt | Node Parser.js

For the readable stream, we can also subscribe to its data and end events to get the block and get notified when the stream is exhausted, as in the classic socket example:

Req.on (' Connect ', function (res, socket, head) {    socket.on (' Data ', function (chunk) {      Console.log ( Chunk.tostring ());    });    Socket.on (' End ', function () {      proxy.close ();    });  });

Switching of readable flow state

It is important to note that the readable stream has two states: flowing mode (torrent) and pause mode (pause). The former can not stop, who was the pipe on the immediately stop to give; the latter pauses until the downstream explicitly calls the Stream.read () request to read the data block. The readable stream is initialized with pause mode.

These two states can be switched between each other, where,

Pause to flowing for any of the following behaviors:

• Add a data event subscription to the readable stream
• Calls to readable. Resume () to open flowing explicitly
• Call the. Pipe (writable) of the readable stream, bridge to a writable stream

Flowing back to pause for any of the following actions:

• The readable stream is not yet pipe to any flow, adjustable. Pause () paused
• The readable stream is already pipe to the stream, remove all data event subscriptions, and call the. Unpipe () method to remove the relationship to the downstream flow

Calmness

In conjunction with the asynchronous nature of the stream, I can write an application that directly bridges the output of user A to the page of User B:

Router.post ('/post ', function (req, res) {    var destination = req.headers[' destination '];//Send to who    cache[ Destionation] = req;    Yes, does not return, so it is best to be an AJAX request});

　　

When User B requests:

Router.get ('/inbox ', function (req, res) {    var user = req.headers[' user '];    Cache.find (user, function (err, previousreq) {//finds the previously saved req       var form = new multiparty. Form ();       Form.parse (previousreq);  There's a file for me       form.on (' part ', function (part) {            part.pipe (res);//streaming Dafa good:            ) part.on (' Error ', function (err) {                Console.log (err);                Messaging.setrequestdone (UniqueID);                return Res.end (ERR);});});    

　　

Reference

1. How to write node programs with Streams:stream-handbook

More articles please visit my blog new address: http://www.moye.me/

Stream in [node. js] node. js