JavaScript asynchronous programming: asynchronous data collection method

First, we try to solve this problem without using any tool functions. The simplest method I can think of is to run the next readFile because of the callback of the previous readFile, track the number of calls that have been triggered so far, and finally display the output. The following is the implementation result of the author .... SyntaxHighlighter. all ();

First, we try to solve this problem without using any tool functions. The simplest method I can think of is to run the next readFile because of the callback of the previous readFile, track the number of calls that have been triggered so far, and finally display the output. The following is the implementation result of the author.

Asyncjs/seriesByHand. js

Var fs = require ('fs ');
Process. chdir ('referes'); // change the working directory
Var concatenation = '';

Fs. readdir ('.', function (err, filenames ){
If (err) throw err;

Function readFileAt (I ){
Var filename = filenames [I];
Fs. stat (filename, function (err, stats ){
If (err) throw err;
If (! Stats. isFile () return readFileAt (I + 1 );

Fs. readFile (filename, 'utf8', function (err, text ){
If (err) throw err;
Concatenation + = text;
If (I + 1 === filenames. length ){
// All files have been read and output can be displayed.
Return console. log (concatenation );
}
ReadFileAt (I + 1 );
});
});
}
ReadFileAt (0 );
}); As you can see, the asynchronous version has more code than the synchronous version. If you use the filter and forEach synchronization methods, the number of lines in the Code is about half, and reading is much easier. It would be nice if these beautiful iterators have asynchronous versions! This can be done with Async. js!

 

When can I throw it?

As you may have noticed, in the code example above, the author ignores the suggestions he made in section 1.4: throwing an exception from the callback is a bad design, especially in the finished environment. However, if an exception is thrown directly in a simple example like this, there is no problem at all. If a code error occurs, throw shut down the code and provides a nice stack trace to explain the cause of the error.

What is really inappropriate here is that the same error processing logic (that is, if (err) throw err) repeats up to three times! In section 4.2.2, we will see how Async. js helps reduce such duplication.

Function Syntax of Async. js
We want to replace the filter and forEach methods used by the synchronous iterator with the corresponding Asynchronous Method. Async. js gave us two options.

Async. filter and async. forEach, which process the given array in parallel.
Async. filterSeries and async. forEachSeries, which process the given array in sequence.
It should be faster to run these asynchronous operations in parallel. Why should we use a sequential method? There are two reasons.

The sequence of workflows is unpredictable. We can store the results as an array and then use the join array to solve this problem. However, there is an additional step.
The maximum number of files that Node and any other application process can read at the same time. If this limit is exceeded, the operating system reports an error. If you can read files in sequence, you do not need to worry about this restriction.
So let's talk about async. forEachSeries first. The following uses the data collection method of Async. js to directly rewrite the Code Implementation of the synchronous version.

Asyncjs/forEachSeries. js

Var async = require ('async ');
Var fs = require ('fs ');
Process. chdir ('referes'); // change the working directory
Var concatenation = '';

Var dirContents = fs. readdirSync ('.');

Async. filter (dirContents, isFilename, function (filenames ){
Async. forEachSeries (filenames, readAndConcat, onComplete );
});

Function isFilename (filename, callback ){
Fs. stat (filename, function (err, stats ){
If (err) throw err;
Callback (stats. isFile ());
});
}

Function readAndConcat (filename, callback ){
Fs. readFile (filename, 'utf8', function (err, fileContents ){
If (err) return callback (err );
Concatenation + = fileContents;
Callback ();
});
}

Function onComplete (err ){
If (err) throw err;
Console. log (concatenation );
} Now our code is beautifully divided into two parts: Task Overview (in the form of async. filter call and async. forEachSeries call) and implementation details (represented by two iterator functions and one completion callback onComplete ).

Filter and forEach are not only Async. js tool functions that correspond to standard functional iteration methods. Async. js also provides the following methods:

Reject/rejectSeries, which is the opposite of filter;
Map/mapSeries, transformation;
Reduce/reduceRight, gradually changing the value;
Detect/detectSeries, and find the value matched by the filter;
SortBy generates an ordered copy;
Some, test whether at least one value meets the given criteria;
Every to test whether all values meet the given criteria.
These methods are the essence of Async. js, allowing you to execute common iterations with the lowest code repetition. Before continuing to explore more advanced methods, let's take a look at the error handling techniques of these methods.

Error Handling Technology of Async. js
It's strange that Node's fs. exists should be the first to start this journey! This also means that the iterator Using Async. js data collection methods (filter/filterSeries, reject/rejectSeries, detect/detectSeries, some, and every) cannot report errors.

For all non-Boolean Async. js iterators, passing the non-null/undefined value as the first parameter of the iterator callback will immediately call the completion callback because of this error value. This is exactly why readAndConcat can work without throw.

Asyncjs/forEachSeries. js

Function readAndConcat (filename, callback ){
Fs. readFile (filename, 'utf8', function (err, fileContents ){
If (err) return callback (err );
Concatenation + = fileContents;
Callback ();
});
} So if the callback (err) is indeed called in readAndConcat, the err will be passed to the completion callback (onComplete ). Async. js is only responsible for ensuring that onComplete is called only once, whether it is called because of the first error or successfully completed all operations.

Asyncjs/forEachSeries. js

Function onComplete (err ){
If (err) throw err;
Console. log (concatenation );
} Node error handling conventions for Async. js data collection methods may not be ideal, but for Async. for all other methods of js, following these conventions can allow errors to flow smoothly from each task to completion callback. In the next section, we will see more such examples.