Process Control in Node. js (1)

The only challenge for programming in an asynchronous framework such as node is how to control which functions are executed in sequence and which functions are executed in parallel. Node does not have a built-in control method. Here I will share some of the skills used to compile the program on this site.

Parallel VS Sequence

Generally, some steps in an application can run only after the results of previous operations are obtained. This is easy to solve in normal sequential execution programs, because each part must wait until the execution of the previous part is completed.

In Node, this problem exists in other methods except for those that execute blocking IO. For example, scan folders, open files, read file content, and query databases.

For my blog engine, some files are organized in a tree structure and need to be processed. The procedure is as follows:

◆ Get the article list (the author's blog uses the file system to store the article list, starting with scanning folders ).

◆ Read and parse the article data.

◆ Obtain the author list.

◆ Read and parse the author's data.

◆ Get the HAML template list.

◆ Read all HAML templates.

◆ Obtain the resource file list.

◆ Read all resource files.

◆ Generate the article html page.

◆ Generate the author page.

◆ Create an index page ).

◆ Generate the feed page.

◆ Generate static resource files.

As you can see, some steps can be executed independently (but some cannot) without other steps ). For example, you can read all files at the same time, but you must scan the folder to obtain the file list. I can write all files at the same time, but I have to wait until the calculation of all the file content is complete before writing.

Use group counters

For the following example of scanning the cat folder and reading the files, we can use a simple counter:

 
 

  
  
var fs = require('fs');  
  
  
 
  
  
fs.readdir(".", function (err, files) {  
  
  
  var count = files.length,  
  
  
      results = {};  
  
  
  files.forEach(function (filename) {  
  
  
    fs.readFile(filename, function (data) {  
  
  
      results[filename] = data;  
  
  
      count--;  
  
  
      if (count <= 0) {  
  
  
        // Do something once we know all the files are read.  
  
  
      }  
  
  
    });  
  
  
  });  
  
  
});  
 
 

Nested callback functions are a good way to ensure their sequential execution. Therefore, in the readdir callback function, we set a countdown counter based on the number of files. Then we execute the readfile operation on each file, which will be executed in parallel and completed in any order. The most important thing is that when each file is read, the counter value is reduced by 1. When the value of the counter is changed to 0, we know that all the files have been read.

Avoid excessive nesting by passing callback Functions

After obtaining the file content, we can perform other operations in the function at the innermost layer. However, when the number of sequential operations exceeds 7, this will soon become a problem.

Let's modify the above instance by passing the callback:

 
 

  
  
var fs = require('fs');  
  
  
 
  
  
function read_directory(path, next) {  
  
  
  fs.readdir(".", function (err, files) {  
  
  
    var count = files.length,  
  
  
        results = {};  
  
  
    files.forEach(function (filename) {  
  
  
      fs.readFile(filename, function (data) {  
  
  
        results[filename] = data;  
  
  
        count--;  
  
  
        if (count <= 0) {  
  
  
          next(results);  
  
  
        }  
  
  
      });  
  
  
    });  
  
  
  });  
  
  
}  
  
  
 
  
  
function read_directories(paths, next) {  
  
  
  var count = paths.length,  
  
  
      data = {};  
  
  
  paths.forEach(function (path) {  
  
  
    read_directory(path, function (results) {  
  
  
      data[path] = results;  
  
  
      count--;  
  
  
      if (count <= 0) {  
  
  
        next(data);  
  
  
      }  
  
  
    });  
  
  
  });  
  
  
}  
  
  
 
  
  
read_directories(['articles', 'authors', 'skin'], function (data) {  
  
  
  // Do something  
  
  
});  
 
 

Now we have written a hybrid asynchronous function that receives some parameters (in this example, the path) and a callback function called after all the operations are completed. All operations will be completed in the callback function. The most important thing is to convert multi-layer nesting into a non-nested callback function.

Combo Library

I used my free time to write a simple Combo library. Basically, it encapsulates the process of counting events and calling the callback function after all events are completed. It also ensures that the callback functions are executed in the registered order regardless of the actual execution time.

 
 

  
  
function Combo(callback) {  
  
  
  this.callback = callback;  
  
  
  this.items = 0;  
  
  
  this.results = [];  
  
  
}  
  
  
 
  
  
Combo.prototype = {  
  
  
  add: function () {  
  
  
    var self = this,  
  
  
        id = this.items;  
  
  
    this.items++;  
  
  
    return function () {  
  
  
      self.check(id, arguments);  
  
  
    };  
  
  
  },  
  
  
  check: function (id, arguments) {  
  
  
    this.results[id] = Array.prototype.slice.call(arguments);  
  
  
    this.items--;  
  
  
    if (this.items == 0) {  
  
  
      this.callback.apply(this, this.results);  
  
  
    }  
  
  
  }  
  
  
};  
 
 

If you want to read data from databases and files and perform some operations after completion, you can do the following:

 
 

  
  
// Make a Combo object.  
  
  
var both = new Combo(function (db_result, file_contents) {  
  
  
  // Do something  
  
  
});  
  
  
// Fire off the database query  
  
  
people.find({name: "Tim", age: 27}, both.add());  
  
  
// Fire off the file read  
  
  
fs.readFile('famous_quotes.txt', both.add());  
 
 

Database Query and file reading will start at the same time. When all of them are completed, the callback function passed to the combo constructor will be called. The first parameter is the database query result, and the second parameter is the file read result.

Conclusion

Tips described in this article:

◆ Obtain the behavior of sequential execution through nested callback.

◆ Obtain the behavior of parallel execution through direct function calls.

◆ Use the callback function to resolve the nesting caused by sequential operations.

◆ Use counters to check when a group of parallel operations are completed.

◆ Use libraries like combo to simplify operations.