Notes on learning Node. js module mechanism

I. CommonJS module specification

Relationship between Node and browser, W3C organization, CommonJS organization, and ECMAScript
Node has implemented a set of module systems based on the CommonJS Modules specification. Let's take a look at the CommonJS module specifications.
CommonJS defines modules in a simple way and consists of three parts: module reference, module definition, and module identification.
1. Module reference
The sample code referenced by the module is as follows:
Var math = require ('math ');
In the CommonJS specification, The require () method exists. This method accepts the Module ID, and introduces the API of a module to the current context.
2. Module definition
In the module, the context provides the require () method to introduce external modules. Corresponding to the introduced function, the context provides the exports object for exporting methods or variables of the current module, and it is a unique export exit. There is also a module object in the module, which represents the module itself, and exports is the attribute of the module. In Node, a file is a module. You can mount the method on the exports object as a property to define the export method:

// Math. js
Exports. add = function (){
Var sum = 0, I = 0, args = arguments, l = args. length;
While (I <l) {sum + = args [I ++];}
Return sum;
};
In another file, after the module is introduced using the require () method, the defined attributes or methods can be called:
1
2
3
// Program. js
Var math = require ('math ');
Exports. increment = function (val) {return math. add (val, 1 );};
3. Module ID
The module identifier is actually a parameter passed to The require () method. It must be a string that matches the name of a small camper, a relative path starting with..., or an absolute path. It can have no file name suffix. js. The module definition is very simple, and the interface is very simple. It limits the methods and variables of clustering to private scopes, and supports the introduction and export functions to smoothly connect upstream and downstream dependencies. Each module has independent space and does not interfere with each other. It also seems clean when being referenced.
II. Node Module Implementation
Node is not fully implemented in accordance with the specifications, but a certain trade-off is made for the module specifications, and a few features are also added. Although the exports, require, and module specifications sound simple, you need to know exactly what the Node has experienced when implementing them.
The following three steps are required to introduce modules in Node.
1. Path analysis
2. File location
3. Compile and execute
In Node, modules are divided into two categories: one is the module provided by Node, which is called the core module; the other is the module written by the user, which is called the file module.
• The core module is compiled into a binary execution file during Node source code compilation. When the Node process is started, some core modules are directly loaded into the memory. Therefore, when these core modules are introduced, the two steps of file locating and compilation can be omitted, in addition, it is prioritized in path analysis, so its loading speed is the fastest.
• The File Module is dynamically loaded at runtime and requires complete path analysis, file locating, and compilation and execution processes, which is slower than the core module.
1. Preferentially load data from the cache
Like the front-end browser that caches static script files to improve performance, Node caches the introduced modules to reduce the overhead during secondary introduction. The difference is that the browser only caches files, while Node caches compiled and executed objects. Whether it is the core module or the file module, the require () method takes the cache priority for secondary loading of the same module, which is the first priority. The difference is that the core module's cache check is prior to the file module's cache check.
2. Path analysis and file location
There are several Identifiers. For different identifiers, the module search and positioning vary to different degrees.
1) module identifier analysis
Node searches for modules based on a module identifier. Module identifiers are mainly divided into the following types in Node.
Core modules, such as http, fs, and path.
Or the relative path File Module starting.
The absolute path File Module starting.
Non-Path File modules, such as custom connect modules.
• Core modules
The priority of the core module is second only to cache loading. It has been compiled into binary code during Node source code compilation, and the loading process is the fastest. If you try to load a custom module with the same identifier as the core module, it will not succeed. If you have compiled an http user module and want to load it successfully, you must select a different identifier or use a path.
• Path-based file module
Identifiers starting with "..." and "/" are processed as file modules. When analyzing the path module, the require () method converts the path to a real path and uses the real path as an index to store the compiled results in the cache, to make the secondary loading faster. Because the file module specifies the exact file location for Node, it can save a lot of time in the search process and its loading speed is slower than that of the core module.
• Custom Module
A custom module is neither a core module nor a path identifier. It is a special file module, which may be in the form of a file or package. Searching for such modules is the most time-consuming and the slowest of all methods.
2) file location
The optimization policy of loading from the cache makes the process of path analysis, file location, and compilation and execution unnecessary during the secondary introduction, greatly improving the efficiency of the module re-loading. However, some details need to be noted during file locating, including analysis of file extensions, directory and package processing.
• File extension analysis
The CommonJS module specification also allows the identifier to not contain the file extension. In this case, the Node will supplement the extension in the order of. js,. json, and. node and try it in sequence. During the attempt, the fs module needs to be called to determine whether the file exists in a synchronous blocking manner. Because Node is single-threaded, this is a cause of performance problems. Tips: if it is a. node or. json file, adding an extension to the identifier passed to require () will speed up a little.
• Directory analysis and packages
In the process of analyzing the identifier, require () may not find the corresponding file after analyzing the file extension, but get a directory. In this case, Node will treat the directory as a package.
In this process, Node provides support for the CommonJS package specification to a certain extent. First, Node searches for the package in the current directory. json (the package description file defined in the CommonJS package specification), through JSON. parse () parses the package description object and extracts the file name specified by the main attribute for locating. If the file name does not have an extension, the file name is analyzed. If the file name specified by the main attribute is incorrect or there is no package. json file at all, Node regards index as the default file name and searches for index. js, index. node, and index. json in sequence.
If no file is successfully located during directory analysis, the custom module goes to the next module path for search. If the module path array is traversed and the target file is not found, an error occurred while searching.
3). Module compilation
In Node, each file module is an object and its definition is as follows:

Function Module (id, parent ){
This. id = id;
This. exports = {};
This. parent = parent;
If (parent & parent. children ){
Parent. children. push (this );
  } 
This. filename = null;
This. loaded = false;
This. children = [];
}
Compilation and execution are the last phase of introducing the file module. After a specific file is located, Node creates a module object and loads and compiles the object according to the path. Different file extensions have different loading methods, as shown below.
•. Js file.
Use the fs module to synchronously read files and compile and execute them.
•. Node file.
This is an extension file written in C/C ++. It loads the final compiled file through the dlopen () method.
•. Json file.
After synchronously reading files through the fs module, parse the returned results using JSON. parse.
• Other file extensions.
They are all loaded as. js files.
Each compiled Module caches its file path as an index on the Module. _ cache object to improve the performance of secondary introduction.
JavaScript module compilation
Back to the CommonJS module specification, we know that each module file contains three variables: require, exports, and module, but they are not defined in the module file. So where does it come from? Even in the Node API documentation, we know that there are two variables _ filename and _ dirname in each module. Where did they come from? If we put the process of directly defining a module on the browser side, global variables may be contaminated.
In fact, during the compilation process, Node wraps the obtained JavaScript file content at the beginning and end. Added (function (exports, require, module, _ filename, _ dirname) {\ n at the end });. A normal JavaScript file will be packaged as follows:

6
(Function (exports, require, module, _ filename, _ dirname ){
Var math = require ('math ');
Exports. area = function (radius ){
Return Math. PI * radius;
};
});
In this way, the scope of each module file is isolated. The encapsulated code will be executed through the runInThisContext () method of the native vm module (similar to eval, but with a clear context and no global pollution) and return a specific function object. Finally, set the exports attribute of the current module object, the require () method, and the module (module object itself ), and the complete file path and file directory obtained in the file location are passed to this function () for execution as parameters.
3. Packages and NPM
In addition to modules, packages and NPM are a mechanism to associate modules.

The definition of CommonJS package specification is also very simple. It consists of the package structure and package description file. The former is used to organize various files in the package, the latter is used to describe the package information for external reading and analysis.
1. Package structure
The package contains a file in the. Zipor tar.gz format. After installation, decompress the file and restore it to a directory. The package directories that fully comply with CommonJS specifications should contain the following files.
Package. json: package description file.
Bin: Directory used to store executable binary files.
Lib: Directory used to store JavaScript code.
Doc: Directory used to store documents.
Test: code used to store unit test cases.
2. Package description file
The package description file is used to express non-code-related information. It is a JSON file named package. json, which is located in the root directory of the package and is an important component of the package. All NPM behaviors are closely related to the fields in the package description file.
For details, refer to the NPM official website's definition specifications for package. json.
You can use npm adduser and npm publish to upload your package to the npm repository.
III. Digression: AMD, CMD, class library compatible with various module specifications
1. AMD
Is an extension of the CommonJS module specification. Its module definition is as follows:
Define (id ?, Dependencies ?, Factory );
2. CMD

3. Compatibility
To enable the same module to run on the frontend and backend, you must consider compatibility with the frontend and implement a standardized module environment. To maintain consistency between the front and back ends, the class library developer needs to wrap the class library code in a closure. The following code defines the hello () method to different runtime environments. It is compatible with Node, AMD, CMD, and common browser environments: