Introduction to NPM module installation mechanism

NPM is the module manager of Node and is extremely powerful. It is one of the important reasons for Node's success.

Because of NPM, we can install modules that others have written by just one line of command.

$ npm install 

This article describes the details of the installation mechanism of the NPM module and how to resolve the problem of slow installation.

First, from the NPM install.

The npm Install command installs the module to the node_modules directory.

$ npm install <packageName>

Prior to installation, the npm install specified module is checked before the node_modules directory exists. If it does exist, it is no longer reinstalled, even if the remote repository already has a new version.

If you want a module to be forced to reinstall regardless of whether it is installed or not, you can use the -f or --force parameters.

$ npm install <packageName> --force

Second, NPM update

If you want to update an installed module, use the npm Update command.

$ npm update <packageName>

It first queries the remote repository for the latest version and then queries the local version. If the local version does not exist, or if the remote version is newer, it will be installed.

Third, registry

npm updateHow does the command know the latest version of each module?

The answer is that the NPM module repository provides a query service called registry. Take npmjs.org as an example, its query service URL is https://registry.npmjs.org/.

After this URL is followed by the module name, you get a JSON object, which is the information for all versions of the module. For example, to access https://registry.npmjs.org/react, you will see information about all versions of the React module.

It has the same effect as the command below.

$ npm view react# npm view 的别名$ npm info react$ npm show react$ npm v react

After the module name of the registry URL, you can also follow the version number or tag to query for a specific version of the information. For example, visit https://registry.npmjs.org/react/v0.14.6, you can see the react version 0.14.6.

Inside the returned JSON object, there dist.tarball is a property that is the URL of the version of the zipped package.

dist: { shasum: ‘2a57c2cf8747b483759ad8de0fa47fb0c5cf5c6a‘, tarball: ‘http://registry.npmjs.org/react/-/react-0.14.6.tgz‘ },

To this URL to download the compressed package, the local decompression, you get the module source code. npm installand npm update commands, are installed in this way module.

Iv. Cache Directory

npm installor npm update command, after downloading the compressed package from registry, it is stored in the local cache directory.

This cache directory, in Linux or MAC default is the directory under the user home directory .npm , in Windows default is %AppData%/npm-cache . You can view the specific location of this directory by configuring the command.

$ npm config get cache$HOME/.npm

You'd better look through this directory.

$ ls ~/.npm # 或者$ npm cache ls

You will see a large number of modules stored inside, and the storage structure is {cache}/{name}/{version} .

$ NPM Cache LS react~/.npm/React/React/0.14.6/~/.npm/React/React/0.14. 6/package.~/.npm/react//0.14. 6/package/~ /.npm/react/react/< Span class= "lit" >0.14. 6/package/package.< Span class= "PLN" >json              

Each version of each module has its own subdirectory, which is the code's compressed package package.tgz file and a description file package/package.json .

In addition, a file is generated {cache}/{hostname}/{path}/.cache.json . For example, a file is generated when the React module is downloaded from the NPM official repository registry.npmjs.org/react/.cache.json .

This file holds the information for all versions, as well as the last modified time of the module and the ETAG returned by the server at the latest request.

{  "time":{ "modified":"2016-01-06T23:52:45.571Z", // ... }, "_etag":"\"7S37I0775YLURCFIO8N85FO0F\""}

For some operations that are not critical (such as npm search or npm view ), NPM will first look .cache.json at the last update time of the module inside, and the gap with the current time is not within an acceptable range. If yes, no more requests are made to the remote repository, but .cache.json the data is returned directly.

.npmThe directory holds a large number of files, and the command to empty it is as follows.

$ rm -rf ~/.npm/*# 或者$ npm cache clean

Five, the installation process of the module

To summarize, the node module installation process is like this.

1. Issue NPM Install command
2. NPM queries the URL of the module compression package to the registry
3. Download the compressed package, stored in the ~/.NPM directory
4. Unzip the package into the Node_modules directory of the current project

Note that after a module is installed, two copies are actually saved locally. One is the ~/.npm compressed package in the directory, and the other is the node_modules extracted code under the directory.
However, npm install when running, only the directory is checked node_modules , not the ~/.npm directory. In other words, if a module ～/.npm has a compressed package under it, but is not installed in the node_modules directory, NPM will still download a new package from the remote repository.

This behavior is guaranteed to always get the latest code, but sometimes it's not what we want. The biggest problem is that it can greatly affect the speed of installation. Even if a module's compression package is in the cache directory, but also to the remote repository to download, how can this not be slow?

In addition, there are occasions where there are no networks (such as on airplanes), but the modules you want to install are clearly in the cache directory and cannot be installed.

Six 、--cache-min parameters

To address these issues, NPM provides a --cache-min parameter for installing modules from the cache directory.

--cache-minThe parameter specifies a time (in minutes) and only modules that exceed this time will be downloaded from registry.

$ npm install --cache-min 9999999 <package-name>

The above command specifies that only more than 999,999 minutes of modules will be downloaded from registry. is actually specified, all modules are installed from the cache, which greatly speeds up the download speed.

It also has another way of writing.

$ npm install --cache-min Infinity <package-name>

However, this is not equivalent to offline mode, which still requires a network connection. Because there are --cache-min some problems with the implementation now.

(1) If the specified module is not in the cache directory, NPM will connect to registry and download the latest version. This is fine, but if you specify the module in the cache directory, NPM will also Connect registry, issuing the etag of the specified module, the server returns a status code of 304, indicating that no need to re-download the package.
(2) If a module is already in the cache, but the version is lower than required, NPM will directly error , instead of going to registry to download the latest version.

The NPM team knows that there are problems and is rewriting the cache. Also, a parameter will be provided in the future --offline so that NPM can be used offline.

However, these improvements do not have a schedule. Therefore, the current use --cache-min of improved installation speed, there is a problem.

Seven, offline installation solutions

The community has put forward several solutions for the offline use of NPM. They can greatly speed up the installation of the module.
The solution is broadly divided into three categories.

First class, Registry agent.

• Npm-proxy-cache
• local-npm( usage )
• Npm-lazy

The usage of the above three modules is very similar, all is a Registry service in this machine, all npm install commands must pass this service proxy.

# Npm-proxy-cache$ npm--proxy http://localhost:8080 \  --https-proxy http< Span class= "pun" >://localhost:8080 \ ---ssl false install # local-npm$ npm set Registry Http://127.0.0.1:5080# npm-lazy $ npm --registry http://localhost:8080/ Install Socket.io               

With the registry service, you can fully implement the cache installation, which can be used offline.

Second class, npm install replace.

If you can change npm install the behavior, you can implement the cache installation. The npm-cache tool is the idea. Wherever used npm install , the alternative can be used npm-cache .

$ npm-cache install

The third class, node_modules as the cache directory.

The idea of this scenario is that instead of using the. NPM cache, the project's Node_modules directory is used as the cache.

• Freight
• Npmbox

The above two tools, can be the project node_modules directory into a compressed package, at a later time, from the compressed package to remove files.
Finish

Introduction to NPM module installation mechanism