Unix Pipes to Javascript Pipes

Unix Pipes

UNIX Pipeline Scanning manuscript

Simple examples:

$ NETSTAT-APN | grep 8080

I believe this is a common use, and here we are not going to elaborate.

So what is the basic idea of plumbing?

• Let each program do only one thing and do it ( do one thing and do it well ), complete a new task, create a new program instead of adding new features to the old program
• Standardize the input and output of each program so that any standard-compliant program can be strung together ( write programs to work together )
• Design and create software, not architecture or systems

NodeJS Stream

Nodejs introduces flow concepts to solve I/O async problems, and if there is no stream, we might want to write code like this:

var http = require(‘http‘);var fs = require(‘fs‘);// 著名的回调地狱又来了var server = http.createServer(function (req, res) { fs.readFile(__dirname + ‘/data.txt‘, function (err, data) { res.end(data); });});server.listen(8000);

But with the stream, we can write the code more gracefully:

var http = require(‘http‘);var fs = require(‘fs‘);var server = http.createServer(function (req, res) { var stream = fs.createReadStream(__dirname + ‘/data.txt‘); // 把两个管子接起来 stream.pipe(res);});server.listen(8000);

Want to add a new feature? Want to compress files with gzip? OK, you can write this:

var http = require(‘http‘);var fs = require(‘fs‘);var oppressor = require(‘oppressor‘);var server = http.createServer(function (req, res) { var stream = fs.createReadStream(__dirname + ‘/data.txt‘); // 将三个管子连起来 stream.pipe(oppressor(req)).pipe(res);});server.listen(8000);

Back-end to Font-end

With the EventStream, Gulp, Webpack pipeline in the back-end of the prevailing, the pipeline gradually from the back end to the front-end infiltration, such as angular, vuejs. We DB.core use the pipeline filter () mode extensively in the Q.js, highly scalable data-tier components Pipe-And-Filter .

(Core of Webpack: Pipeline plug-in)

So what about the front-end plumbing? Let's take DB.core a look at the example.

Pipes in Db.core

DBis designed to solve the common problem of CGI pull, such as: distinguishing between correct and wrong logic, general error logic processing, and general processing of the login state. But DB it is difficult to reuse, because we find that each business has a different commonality, usually we start a new business is always to be DB copied down to make a lot of intrusive changes to apply to the new business requirements. And in 齐齐互动视频 , we like the pipeline filter pattern refactoring our DB . For details, see: https://github.com/miniflycn/db

Where's the problem?

• Rule of simplicity:design for simplicity; Add complexity only where you must.
• Rule of Parsimony:write A big program only if it is clear by demonstration this nothing else would do.

Let DB 's do a lot of things, cause every time you do one thing, insert various codes. With the pipeline filter mode, everything is split into separate filters, one for each filter, but a single thing to the extreme.

The single function principle (responsibility principle) stipulates that each class should have a single function, and that the function should be fully encapsulated by this class. All its services (this class) should be closely parallel to the function (functionally parallel, meaning no dependencies).

Martin defines the function as "the reason for the change" and summarizes why a class or module should have and only one change. A concrete example is imagining a module for editing and printing a report. There are two reasons for such a module to change. First, the contents of the report can be changed (edited). Second, the format of the report can be changed (printed). These changes occur because of a completely different cause: one is the modification of the essence, and the other is the modification of the surface. The single-function principle argues that these two aspects of the problem are actually two separate functions, so they should be separated in different classes or modules. The design of coupling things that have different reasons for change is bad.

--from Wiki

Basic implementation of pipeline filter mode used in front-end

    /**     * _apply     * @param {Array} handles 处理函数列队，每一个是一个管子     * @param {*} data 要处理的数据     * @param {Object} options 可选参数     * @param {Function} cb 处理后回调 */ _apply: function (handles, data, options, cb) { var i = 0, l = handles.length, res = data; for (i; i < l; i++) { res = handles[i].call(this, res, options); // if handle return false, just break if (res === false) return; } cb(res); },

Db.core Complete Implementation

var $ =Require' jquery ');/** * DB *@class *@param {Object} options This is just a $.ajax setting *@param {Array} options.errhandles *@param {Array} options.succhandles *@param {Function} OPTIONS.SUCC *@param {Function} Options.err */function DB (options) {This._init (options);} $.extend (Db.prototype, {_init:function (options) {This.errhandles = Options.errhandles | | [];This.succhandles = Options.succhandles | | [];This.errHandles.unshift.apply (This.errhandles, DB.options.errHandles | | []);This.succHandles.unshift.apply (This.succhandles, DB.options.succHandles | | []); Options = $.extend ({}, Db.options | | {}, options);this.options = options; }, _wrap:function (options) {VarSelf =This Options.success =function (data) {Want to modify the line for judging error or success Data.retcode = = =2}Self._apply (Self.succhandles, data, Options, OPTIONS.SUCC):Self._apply (Self.errhandles, data, Options, Options.err); }; Options.error =function (data) {Self._apply (Self.errhandles, data, Options, Options.err); };return options; }, _apply:function (handles, data, options, CB) {var i =0, L = handles.length, res = data;for (i; I < L; i++) {res = Handles[i].call (This, res, options);If handle return false, just breakif (res = = =Falsereturn;} CB (RES); }, /** * Ajax * @param {Object} options This is just a $.ajax setting *  @param {function} OPTIONS.SUCC *  @param {Function} Options.err */ajax: function" (options) {options = this._wrap ($.extend ({}, function (options) {var DB = new DB (options); return function (opt) {db.ajax (opt); return this;}, Extend:jQuery.extend, // Default options options: {}}); module.exports = DB;           

This allows us to put all of the previous business data detection, even data assembly, in Succhandles and Errhandles. This is an example of what we use in the alignment project, and we can see that we put the data assembly on the DB layer.

Unix Pipes to Javascript Pipes