Implementation of node. js BigPipe _ node. js

This article mainly introduces node. for details about js Implementation of BigPipe, BigPipe is a technology developed by Facebook to optimize webpage loading speed. The core concept of BigPipe is to use only one HTTP request, but the page elements are not sent in order, for more information, see BigPipe, a technology developed by Facebook to optimize webpage loading speed. Almost no articles are implemented using node. js on the Internet. In fact, BigPipe is rarely implemented on the Internet, not just node. js. So long after this technology came into being, I thought that the whole web page framework was sent first, and then requested the module in the page with another or several ajax requests. Not long ago, I realized that the core concept of BigPipe was to use only one HTTP request, but the page elements were not sent in order.

It is easy to understand this core concept. Thanks to the asynchronous feature of node. js, it is easy to use node. js to implement BigPipe. This article will explain the causes of BigPipe technology and a simple implementation based on node. js step by step.

I will use express for demonstration. For simplicity, we use jade as the template engine, and we do not use the engine subtemplate (partial) feature, the data of the parent template is HTML after the sub-template is rendered.

Create a nodejs-bigpipe folder and write a package. json file as follows:

The Code is as follows:

{
"Name": "bigpipe-experiment"
, "Version": "0.1.0"
, "Private": true
, "Dependencies ":{
"Express": "3. x. x"
, "Initialize lidate": "latest"
, "Jade": "latest"
}
}

Run npm install to install the three libraries. The latest lidate is used to conveniently call jade.

First, make the simplest attempt. There are two files:

App. js:

The Code is as follows:

Var express = require ('express ')
, Cons = require ('invalid lidate ')
, Jade = require ('jade ')
, Path = require ('path ')

Var app = express ()

App. engine ('jade ', cons. jade)
App. set ('view', path. join (_ dirname, 'view '))
App. set ('view engine ', 'jade ')

App. use (function (req, res ){
Res. render ('layout ',{
S1: "Hello, I'm the first section ."
, S2: "Hello, I'm the second section ."
})
})

App. listen (3000)

Views/layout. jade

The Code is as follows:

Doctype html

Head
Title Hello, World!
Style
Section {
Margin: 20px auto;
Border: 1px dotted gray;
Width: 80%;
Height: 150px;
}

Section # s1! = S1
Section # s2! = S2

The effect is as follows:

Next, we will put the two section templates into two different template files:

Views/s1.jade:

The Code is as follows:

H1 Partial 1
. Content! = Content

Views/s2.jade:

The Code is as follows:

H1 Partial 2
. Content! = Content

Add some styles in layout. jade's style.

The Code is as follows:

Section h1 {
Font-size: 1.5;
Padding: 10px 20px;
Margin: 0;
Border-bottom: 1px dotted gray;
}
Section p {
Margin: 10px;
}

Change the app. use () part of app. js:

The Code is as follows:

Var temp = {
S1: jade. compile (fs. readFileSync (path. join (_ dirname, 'view', 's1. jade ')))
, S2: jade. compile (fs. readFileSync (path. join (_ dirname, 'view', 's2. jade ')))
}
App. use (function (req, res ){
Res. render ('layout ',{
S1: temp. s1 ({content: "Hello, I'm the first section ."})
, S2: temp. s2 ({content: "Hello, I'm the second section ."})
})
})

Previously, we said, "the HTML after the sub-template is rendered as the data of the parent template", which means that temp. s1 and temp. the two s2 methods generate the HTML code of the s1.jade and s2.jade files, and then use the two pieces of code as layout. the values of s1 and s2 variables in jade.

The page looks like this:

In general, the data in the two sections is obtained separately-whether by querying the database or RESTful request, we use two functions to simulate such asynchronous operations.

The Code is as follows:

Var getData = {
D1: function (fn ){
SetTimeout (fn, 3000, null, {content: "Hello, I'm the first section ."})
}
, D2: function (fn ){
SetTimeout (fn, 5000, null, {content: "Hello, I'm the second section ."})
}
}

In this way, the logic in app. use () will be more complex. The simplest way to handle this is:

The Code is as follows:

App. use (function (req, res ){
GetData. d1 (function (err, s1data ){
GetData. d2 (function (err, s2data ){
Res. render ('layout ',{
S1: temp. s1 (s1data)
, S2: temp. s2 (s2data)
})
})
})
})

In this way, we can get the expected result, but it takes 8 seconds to return the result.

In fact, the implementation logic shows that getData. d2 is called only after the results of getData. d1 are returned, but they do not have such dependency. We can use libraries such as async for processing JavaScript asynchronous calls to solve such problems, but here we will simply hand-write them:

The Code is as follows:

App. use (function (req, res ){
Var n = 2
, Result = {}
GetData. d1 (function (err, s1data ){
Result. s1data = s1data
-- N | writeResult ()
})
GetData. d2 (function (err, s2data ){
Result. s2data = s2data
-- N | writeResult ()
})
Function writeResult (){
Res. render ('layout ',{
S1: temp. s1 (result. s1data)
, S2: temp. s2 (result. s2data)
})
}
})

This takes only 5 seconds.

Before the optimization, we will add the jquery library and put the css style into an external file. By the way, we will use the runtime required by the browser to use the jade template. js files are also added to include the app. run the following command in the js directory:

The Code is as follows:

Mkdir static
Cd static
Http://code.jquery.com/jquery-1.8.3.min.js-o jquery. js
Ln-s ../node_modules/jade/runtime. min. js jade. js

Put the code in the style label in layout. jade into static/style.css, and change the head label:

The Code is as follows:

Head
Title Hello, World!
Link (href = "/static/style.css? 1.1.9 ", rel =" stylesheet ")
Script (src = "/static/jquery. js? 1.1.9 ")
Script (src = "/static/jade. js? 1.1.9 ")

In app. js, we simulate the download speeds of both of them as two seconds. Before app. use (function (req, res:

The Code is as follows:

Var static = express. static (path. join (_ dirname, 'static '))
App. use ('/static', function (req, res, next ){
SetTimeout (static, 2000, req, res, next)
})

Due to the influence of external static files, the loading time of our page is about 7 seconds.

If we return the head part as soon as we receive the HTTP request, and then the two sections will return after the asynchronous operation is completed, this uses the HTTP multipart Transmission Encoding mechanism. In node. js, the Transfer-Encoding: chunked header is automatically added when the res. write () method is used. In this way, when the browser loads static files, the node server waits for the asynchronous call result. We will delete the two lines in layout. jade:

The Code is as follows:

Section # s1! = S1
Section # s2! = S2

Therefore, {s1 :..., S2 :... } This object, and because res. by default, render () calls res. end (), we need to manually set the callback function after the render is complete, and use res. write () method. Layout. jade content does not need to be included in the writeResult () callback function. We can return the content when we receive this request. Note that we have manually added the content-type header:

The Code is as follows:

App. use (function (req, res ){
Res. render ('layout ', function (err, str ){
If (err) return res. req. next (err)
Res. setHeader ('content-type', 'text/html; charset = UTF-8 ')
Res. write (str)
})
Var n = 2
GetData. d1 (function (err, s1data ){
Res. write ('

'+ Temp. s1 (s1data) +' ')
-- N | res. end ()
})
GetData. d2 (function (err, s2data ){
Res. write (' '+ Temp. s2 (s2data) +' ')
-- N | res. end ()
})
})

Now the loading speed is about five seconds. In actual operation, the browser first receives part of the head code and loads three static files. This takes two seconds, and then to the third second, Part 1 of Partial appears, part 2 appears in 5th seconds, and the webpage loading ends. No. The effect is the same as that of the previous five seconds.

However, you must note that this effect is achieved because getData. d1 is better than getData. d2 is fast. That is to say, the first response to the block in the webpage depends on the asynchronous call result of the interface. If we call getData. if d1 is changed to 8 seconds, the system will first return part 2 of Partial, and the sequence of s1 and s2 is reversed. The final webpage result is inconsistent with our expectation.

This problem eventually leads us to BigPipe. BigPipe is a technology that decouples the display sequence of each part of a webpage from the Data Transmission sequence.

The basic idea is to first transmit the general frame of the entire web page. The content to be transmitted later must be expressed with null p (or other labels:

The Code is as follows:

Res. render ('layout ', function (err, str ){
If (err) return res. req. next (err)
Res. setHeader ('content-type', 'text/html; charset = UTF-8 ')
Res. write (str)
Res. write ('

')
})

Then write the returned data in JavaScript.

The Code is as follows:

GetData. d1 (function (err, s1data ){
Res. write ('script $ ("# s1" cmd.html ("'+ temp. s1 (s1data ). replace (/"/g, '\"') + '") script')
-- N | res. end ()
})

The processing of s2 is similar to this. At this time, you will see that the second of the Request webpage shows two blank dashed boxes, the fifth second shows part of Partial 2, the Eighth Second shows part of Partial 1, and the webpage request is complete.

So far, we have completed a Web page that is implemented by the simplest BigPipe technology.

Note that the webpage fragment to be written contains a script tag, for example, changing s1.jade:

The Code is as follows:

H1 Partial 1
. Content! = Content
Script
Alert ("alert from s1.jade ")

Then, refresh the webpage and you will find that this alert statement is not executed and the webpage has an error. Check the source code and check whether the error is caused by the character string in script. Replace it with <\/script>.

The Code is as follows:

Res. write ('script $ ("# s1" cmd.html ("'+ temp. s1 (s1data ). replace (/"/g ,'\\"'). replace (/<\/script>/g, '<\/script>') + '") script')

The above describes the principle of BigPipe and the basic method for implementing BigPipe using node. js. In practice, how should we use it? The following provides a simple method for you to use. The Code is as follows:

The Code is as follows:

Var resProto = require ('express/lib/response ')
ResProto. pipe = function (selector, html, replace ){
This. write ('script '+' $ ("'+ selector +'"). '+
(Replace = true? 'Replace with': 'html ') +
'("' + Html. replace (/"/g ,'\\"'). replace (/<\/script>/g, '<\/script>') +
'") Script')
}
Function PipeName (res, name ){
Res. pipeCount = res. pipeCount | 0
Res. pipeMap = res. pipeMap || {}
If (res. pipeMap [name]) return
Res. pipeCount ++
Res. pipeMap [name] = this. id = ['pipe', Math. random (). toString (). substring (2), (new Date ()). valueOf ()]. join ('_')
This. res = res
This. name = name
}
ResProto. pipeName = function (name ){
Return new PipeName (this, name)
}
ResProto. pipeLayout = function (view, options ){
Var res = this
Object. keys (options). forEach (function (key ){
If (options [key] instanceof PipeName) options [key] =''
})
Res. render (view, options, function (err, str ){
If (err) return res. req. next (err)
Res. setHeader ('content-type', 'text/html; charset = UTF-8 ')
Res. write (str)
If (! Res. pipeCount) res. end ()
})
}
ResProto. pipePartial = function (name, view, options ){
Var res = this
Res. render (view, options, function (err, str ){
If (err) return res. req. next (err)
Res. pipe ('#' + res. pipeMap [name], str, true)
-- Res. pipeCount | res. end ()
})
}
App. get ('/', function (req, res ){
Res. pipeLayout ('layout ',{
S1: res. pipeName ('s1name ')
, S2: res. pipeName ('s2name ')
})
GetData. d1 (function (err, s1data ){
Res. pipePartial ('s1name', 's1', s1data)
})
GetData. d2 (function (err, s2data ){
Res. pipePartial ('s2name', 's2', s2data)
})
})

Add two sections in layout. jade:

The Code is as follows:

Section # s1! = S1
Section # s2! = S2

The idea here is that the pipe content should first use a span tag placeholder, asynchronously obtain the data and render the corresponding HTML code, and then output it to the browser, replace the placeholder span element with jQuery's replaceWith method.

The code in this article is in the https://github.com/undozen/bigpipe-on-node, I made every step into a commit, I hope you clone to the local actually run and have a look at hack. Because the loading sequence is involved in the next steps, you must open your browser to experience it and cannot see it from the top (in fact, you can use gif animation, but I am too lazy to do it ).

There is still a lot of room for optimization about BigPipe practices. For example, it is best to set a trigger time value for pipe content. If the asynchronous call data Returns quickly, you do not need to use BigPipe, directly generate a webpage and send it out. BigPipe can be used only when the data request exceeds a certain time. Compared with ajax, BigPipe not only saves the number of requests from the browser to the node. js server, but also saves the number of requests from the node. js server to the data source. But the specific optimization and practical methods will be shared after the snowball network uses BigPipe.