Deep parsing of JS Asynchronous Programming tool Generator

When we write the Nodejs program, we often use the callback function to process the returned data after an operation is completed, and I simply understand that it is asynchronous programming.

If there is a lot of action, then the nesting of callbacks is necessary, so if the operation is very large, then the callback nesting will become unbearable.

The promises we know is asked to solve the problem. However, promises is not a new function, it is just a new way of writing, the original lateral development of the callback function, was lined up in the vertical development of the team.

However, unlike generator, it is a new solution.

All the code mentioned in this article can be found here: [View source].

Article directory:

• One, ES6 transcoding device
• Ii. meaning of the generator function and the yield keyword
• Third, why use the generator function?
• Iv. the generator in KOA
• V. About generator easy to misunderstand
• Six, the Shining spot of generator
• Vii. Expansion of Reading

One, ES6 transcoding device

The generator function is an asynchronous programming solution provided by ES6, which is completely different from the traditional functions.

While the support for ES6 is now becoming more and more high for each browser, it is not complete at all, and it is also designed to be compatible with different versions of browsers, so in some cases we may need to convert ES6 code into ES5 standard code with some ES6 transcoding.

1.1 Babeljs-online

I like to use [babeljs-online] when I understand ES6 and write some ES6-related code snippets on the side. With it we can write and run the ES6 code in real time without switching back and forth between the text editor and the command line console.

It's grown this way:

The left can be directly written ES6 syntax code, the lower left corner provides real-time error detection;

On the right is a real-time compilation of ES5 syntax code, the lower right corner can output console.log ();

1.2 Babel-node

Of course we can also install the Babel module:

1 npm Install--global Babel2 Babel-node

So, like in Repl, we can write and execute the ES6 code:

However, Babel-node now does not support multiple lines of input.

1.3 Traceur Online Editor

Traceur is also an online editor that converts ES6 code into ES5 code online.

I prefer Babel's online editor with real-time syntax error reminders and output of running results. In general I choose it to write some code snippets.

Ii. meaning of the generator function and the yield keyword

With some simple snippets, let's take a look at the meaning of generator and some of its key points. You can type your own code in [Babeljs-online] to understand its meaning.

Here's a simple example:

View Code

View running results: [Hellogenerator Code Snippet]

Execution of the 2.1 yield statement pauses the execution of the current function and saves the current stack, returning the value of the current yield statement.

The 2.2 generator function differs from a normal function in that it defines only the walker, does not execute, and each time it invokes the next method of the Walker, it executes from the head of the function body or the last stop, until the next yield statement.

The yield statement is the flag of the pause, and the next method encounters yield, pausing to perform the subsequent operation and returning the value of the expression immediately following the yield as the value of the returned object's Value property. When the next method is called again, it continues to execute down, knowing that the next yield statement is encountered. If you do not encounter a new yield statement, run until the end of the function, the value of the expression following the return statement, as the value of the Value property, and if the function does not have a return statement, the value of the values property is undefined.

2.3 If the generator function does not use the yield statement, it becomes a purely deferred execution function:

View Code

View running results: [generator function for deferred execution]

The 2.4 yield statement cannot be used in the normal function, otherwise it will be an error:

View Code

To view the results of a run: [Example of error using yield statement in a normal function]

You can see the editor error.

The 2.5 yield statement itself has no return value, or always returns undefined. The next method can take a parameter that will be used as the return value of the previous yield statement. However, because the next method parameter represents the return value of the previous yield statement, the first time you use the next method, you cannot have parameters, and V8 directly ignores the parameter when you first use the next method, and the parameter is valid only if you start with the next method for the second time.

The 2.6 For...of loop can automatically traverse the generator function and does not need to call the next method at this time:

View Code

The above code will output 1,2,3,4 and will not output 5, because once the next method returns the object's Done property, the True,forof loop stops.

2.7 If the yield statement is followed by a walker, you need to add an asterisk after the yield command to indicate that it is returning a walker, which is called the yield* statement.

You can look at the following example:

View Code

View Run Results: [yield* statement]

In fact, the yield* statement is equivalent to deploying a for...of loop inside the generator function.

For more details on the Generator function, it is highly recommended to read Nanyi teacher's ES6 Introduction: [ES6 Introduction-Generator]

Third, why use the generator function?

Although we know the generator function, we know its meaning and how to use it. But why use the generator function? What is it that solves the problem? What are the advantages?

We start with a simple node program [to find the largest file in the current directory] and use a different solution to solve the callback problem.

We want to write a module and then call it in the portal file to find the largest file in the current directory:

1 var findlargest = require ('./readmaxnested '); 2 3 4//Gets the maximum file name under the current root directory of 5 findlargest ('./', function(err, FileName) {6    if (err) return console.log (err); 7    console.log (' largest file was: ', filename); 8});  

3.1 Nested Solutions

The first conceivable scenario:

1. Read all files under the current folder;

2. Get stat for each file, compare file size when confirming IO operation is complete;

3. Filter out directories, etc. only compare files;

4. Returns the filename of the largest file via callback.

The implementation should look like this:

View Code

Normal and correct solution, but there are some problems. You can see that we start the file comparison by using the counter variable to ensure that all IO operations are complete, and that the errored variable is used to ensure that only one error return is invoked once an error occurs. This allows you to see that the above program requires extra care when managing parallel operations.

3.2 Modular Solutions

To make the code easier to reuse and test, we can slightly refine the code to extract the available functions and methods:

View Code

However, this has not made some substantial improvements, as you can see, we managed the process manually by two variables.

3.3 Workarounds for using async modules

We use the very popular async module to rewrite our program:

View Code

Here are two key methods:

1.async.waterfall, which provides a series of execution processes, through a series of callback data can be passed from one function to the next function;

2.async.map, you can execute fs.stat in parallel and return an array of results;

You can see that async provides only one callback, and we don't have to worry about the process control and the number of times the callback function is called.

3.4 Solutions for using promises

Promises is a workaround for multiple nested callbacks. It's not a new syntax feature, it's just a new notation. It allows you to change the horizontal loading of the callback function to portrait loading.

Let's rewrite the code using the promises notation:

View Code

Q.all will get the stats of all files in parallel and return an array.

In fact, the code at a glance seems to be somewhat redundant, the original task was promises wrapped a bit, looks like a lot of then, the semantics become somewhat unclear.

More about the Promises and Q modules can be read: [Promises in Nodejs with Q]

3.5 Generator Solutions

We use ES6 's generator, an asynchronous programming solution, to improve the code:

View Code

The Co module and the Thunk function are used here, followed by a detailed explanation, which is skipped first.

We can see a few lines of code that solve the problem subtly, with clear semantics and no problem with callbacks.

We have seen a more elegant and simple way of asynchronous programming that uses Co to encapsulate generator.

You can view all of the above code here: [Find the largest file in the current directory]

Iv. the generator in KOA

As we can see from the last code snippet above, there is a certain difference between the generator and the native generator, because generator in KOA is encapsulated with Co.

Simple use of 4.1 Co :

1 var co = require (' Co '); 2 var fs = require (' FS '); 3  4 function Read (file) {5   return function(fn {6     fs.readfile (file, ' UTF8 '} 9 Co (function *() {ten var a = yield read ('. Gitignore ' console.log (a.len GTH); var b = yield read (' Package.json ' console.log (b.length);       

Co requires all asynchronous functions to be thunk functions:

1 function Read (file) {2   return function(FN) {3     fs.readfile (file, ' UTF8 '  }5} 

If you need to do some processing of the data returned by the Thunk function, it can be written in the callback function:

1 function Read (file) {2   return function(FN) {3     fs.readfile (file, ' UTF8 ', function(err,result) {4         if (err) return FN (err); 5         fn (null }8}    

We can also use the Thunkify module instead of writing the thunk function ourselves:

1 var thunkify = require (' thunkify '); 2 var fs = require (' FS '); 3 4 var read = Thunkify (fs.readfile); 

Get the return result of the thunk function, just use the yield keyword to:

1 var a = yield read ('. Gitignore '); 2 Console.log (a.length);

We can see that we no longer have to use the next method, the Co will generator function of the flow package.

4.2 thunk function

We put the parameters in a temporary function, and then pass the temporary function into the body of the function, when the parameter is used to evaluate the temporary function. This temporary function is called the thunk function.

Thunk is an implementation strategy for "Bogwang call", which is used to replace an expression:

1 function f (m) {2   return m * 2;      } 4  5 f (x + 5); 6  7//equivalent to 8  9 var thunk = function () {   return x + 5};12 function< c13> F (thunk) {thunk return () * 2;      

The JavaScript language is a call to value, and its Thunk function has a different meaning. In the JavaScript language, the Thunk function replaces not an expression, but a multi-parameter function, replacing it with a single-argument version and accepting only the callback function as a parameter:

Fs.readfile (FileName, callback); 3  4//Thunk version of ReadFile (single parameter version) 5 var readfilethunk =readfilethunk (callback); 7  8 var Thunk = function (fileName) {9   return function (callback) {Ten     return };12};   

Read more: [Meaning and usage of the thunk function]

V. About generator easy to misunderstand

Now it looks like all the generator functions are running, and we use the yield keyword just to make it pause. So is this the legendary implementation of multithreading in JS?

But it's not. As long as we firmly remember that JS is really single-threaded. We just have the ability to pause in a function.

Also, if the performance requirements are high, generator may not be suitable as a preferred option.

We can run a CPU-intensive example to see.

The Fibonacci sequence is calculated using the normal function and the generator function, respectively:

1 varSuite =New(Require (' benchmark ')). Suite;2 3 functionfib (n) {4   varCurrent = 0,next = 1, swap;5    for(vari=0;i<n;i++){6Swap = Current,current =Next;7Next = swap +Next;8   }9   returnCurrent ;Ten } One  A function*Fibgen (n) { -   varCurrent = 0,next = 1, swap; -    for(vari=0;i<n;i++){ theSwap = Current,current =Next; -Next = swap +Next; - yield current; -   } + } -  + Suite A. Add (' Regular ',function(){ atFIB (20); -   }) -. Add (' Generator ',function(){ -      for(varN of Fibgen (20)); -   }) -. On (' Complete ',function(){ inConsole.log (' Results: '); -      This. ForEach (function(Result) { to Console.log (result.name,result.count); +     }) -   }) the. Run ()

View Code

Here we use the benchmark module to measure the execution time.

Then we execute:

1 node--harmony fibonacci.js

You can see the results of the execution:

1 Results: 2 Regular 14345833 Generator 39962

You can see that the efficiency of the generator is no better than the normal function (the larger the value the better).

Six, the Shining spot of generator

We have seen so many generator and written some examples, so what is the point of its shining? Let's sum it up.

6.1 Lazy Execution

The legendary lazy evaluation. We can actually do this through closures, but it makes things easier by using yield. We can get it when we need to get the right time:

1 var fibiter = Fibgen (2var )next = fibiter.next ()3  Console.log (next.value)4  5 setTimeout (function  () {6   var next = fibiter.next ()7  console.log (next.value)8 },2000)

If we want to do it from start to finish, we can use the for-for loop as mentioned above:

1  for (var n of Fibgen () {2  console.log (n)3 }

6.2 Unlimited Execution

Since the execution of the generator function is lazy, when we need to execute to get the corresponding value, then we can achieve infinite execution:

1 function* Fibgen () {2   var current = 0, next = 1, swap3   C9>while (true) {4     swap = current, current = next5     next = swap + next6    yieldcurrent7  }8  }

6.3 Synchronized Control Flow

The initial use of the generator for the synchronization control process is Task.js (now no longer exists). This concept became popular because of the Co module and some other promise implementations.

But how is it implemented?

In node, we always put things in the callback to do, to synchronize the effect, which may be a relatively elementary synchronization implementation. There is also an example of the evolution of the generator version of the high efficiency step-by-step.

Here's a simple example that allows us to get a synchronous version, but it's written like an asynchronous implementation:

1 varFS = require (' FS ');2 varThunkify = require (' thunkify '));3 varReadFile =thunkify (fs.readfile);4 5Runfunction* (){6   Try{7     varFILE = Yield readFile ('./fibonacci.js '));8 Console.log (file.tostring ());9}Catch(er) {Ten console.error (er); One   } A }); -  - functionRun (GENFN) { the   varGen =Genfn (); - next (); -   functionNext (er,value) { -     if(er)returnGen.Throw(er); +     varContinuable =Gen.next (value); -  +     if(Continuable.done)return; A     varCBFN =Continuable.value; at CBFN (next); -   } -}

View Code

All the code mentioned in this article can be found here: [View source].

Vii. Expansion of reading:

[ES6 Generator]

[Sync Module]

[Meaning and usage of async functions]

[Meaning and usage of the generator function]

[Meaning and usage of the thunk function]

[Meaning and usage of the CO function]

[Generators in good use]

Deep parsing of JS Asynchronous Programming tool Generator