Node. js advanced programming: using JavaScript to build scalable applications (7) 3.7 files, processes, streams and networks-query and read/write files

Document directory

• Process file path
• Fs module Introduction

For the list of articles in this series and the translation progress, see Node. js advanced programming: using Javascript to build scalable applications (〇)

This document corresponds to the third part of the original article, Chapter 7: Files, Processes, Streams, and Networking: Querying, Reading from, and Writing to Files

The article is copied from Word, and the layout is somewhat inconsistent. You can click here to download the PDF version of this article.

 

Chapter 7: Query and read/write documents This chapter covers:

• Process file path
• Extract information from the file path
• Understanding file descriptors
• Use fs. stat () to obtain file information
• Open, read/write, and close a file
• Prevent file descriptor Leakage

 

Node has a set of data flow APIs to process files as it processes network streams. It is easy to use, but it only allows sequential file processing and cannot read/write files randomly. Therefore, some underlying file system operations are required.

This chapter covers the basics of file processing, including how to open a file, read a part of the file, write data, and close the file.

Many file APIs of Node are almost the opposite of file APIs in UNIX (POSIX), such as using file descriptors, just like in UNIX, the file descriptor is also an integer in Node, representing the index of an object in the process file descriptor table.

There are three special file descriptors-1, 2, and 3. They represent standard input, standard output, and standard error file descriptors. Standard input, as its name implies, is a read-only stream, which is used by a process to read data from the console or process channel. Standard output and standard errors are file descriptors used only to output data. They are often used to output data to the console, other processes, or files. Standard errors are responsible for error information output, while standard output is responsible for general process output.

Once the process is started, these file descriptors can be used. They do not actually have the corresponding physical files. You cannot read or write data at a random location ,(Note:The original Article is You can write to and read from specific positions within the file. according to the context, the author may write less "not"), and can only read and output data in the same order as operating the network data stream. The written data cannot be modified.

Normal files are not subject to such restrictions. For example, in a Node, you can create files that only append data to the end, and create files that read and write random locations.

Almost all file-related operations involve file path processing. This chapter first introduces these tool functions, and then explains file read/write and data operations in depth.

Process file path

File paths are classified into relative paths and absolute paths, which are used to represent specific files. You can merge file paths, extract file name information, and even check whether a file exists.

In Node, you can use strings to process the file path, but this will complicate the problem. For example, if you want to connect different parts of the path, some parts end with "/" but some do not, in addition, path delimiters may be different in different operating systems, so when you connect them, the code will be very cool and troublesome.

Fortunately, Node has a module named path that can help you standardize, connect, parse paths, convert absolute paths to relative paths, and extract information from the paths, checks whether a file exists. In general, the path module only processes strings and does not perform verification in the file system (except for the path. exists function ).

Path Standardization

It is usually a good idea to standardize paths before they are stored or used. For example, the file path obtained by the user input or configuration file, or the path connected by two or more paths should be standardized. You can use the normalize function of the path module to standardize a path, and it can also process "...", "." "//". For example:

                  var path = require('path');                   path.normalize('/foo/bar//baz/asdf/quux/..');                   // => '/foo/bar/baz/asdf'

Connection path

You can use the path. join () function to connect any number of path strings. You only need to pass all path strings to the join () function in sequence:

                   var path = require('path');                   path.join('/foo', 'bar', 'baz/asdf', 'quux', '..');                   // => '/foo/bar/baz/asdf'

As you can see, path. join () automatically standardizes the path.

Resolution path

You can use path. resolve () to parse multiple paths into an absolute path. Its function is like constantly performing "cd" operations on these paths. Unlike the parameters of the cd command, these paths can be files and they do not have to actually exist-path. the resolve () method does not access the underlying file system to determine whether the path exists. It is only a string operation.

For example:

                   var path = require('path');                   path.resolve('/foo/bar', './baz');                   // => /foo/bar/baz                   path.resolve('/foo/bar', '/tmp/file/');                   // => /tmp/file

If the resolution result is not an absolute path, path. resolve () will append the current working directory as the path to the resolution result, for example:

Path. resolve ('wwwroot', 'static _ files/png /','.. /gif/image.gif '); // if the current working directory is/home/myself/node, will return // =>/home/myself/node/wwwroot/static_files/gif/image.gif'

Calculate the relative paths of two absolute paths

Path. relative () tells you to jump from an absolute address to another absolute address, for example:

                var path = require('path');                path.relative('/data/orandea/test/aaa', '/data/orandea/impl/bbb');                // => http://www.cnblogs.com/impl/bbb

Extract data from a path

Take the path "/foo/bar/myfile.txt" as an example. If you want to obtain all contents of the parent directory (/foo/bar) or read other files in the same directory, you must use path. dirname (filePath) obtains the directory of the file path, for example:

                   var path = require('path');                   path.dirname('/foo/bar/baz/asdf/quux.txt');                   // => /foo/bar/baz/asdf

Or you want to get the file name from the file path, that is, the last part of the file path. You can use the path. basename function:

                   var path = require('path');                   path.basename('/foo/bar/baz/asdf/quux.html')                   // => quux.html

The file path may also contain the file extension, which is usually the part of the string after the last "." character in the file name.

Path. basename can also accept an extension string as the second parameter, so that the returned file name will automatically remove the extension and only return the file name:

                   var path = require('path');                   path.basename('/foo/bar/baz/asdf/quux.html', '.html');                   // => quux

To do this, you must first know the file extension. You can use path. extname () to obtain the extension:

                   var path = require('path');                   path.extname('/a/b/index.html');                   // => '.html'                   path.extname('/a/b.c/index');                   // => ''                   path.extname('/a/b.c/.');                   // => ''                   path.extname('/a/b.c/d.');                   // => '.'

Check whether the path exists

So far, the preceding path processing operations have nothing to do with the underlying file system, but only some string operations. However, sometimes you need to determine whether a file path exists. For example, you sometimes need to determine whether a file or directory exists. If it does not exist, you can create it using path. exsits ():

                   var path = require('path');                   path.exists('/etc/passwd', function(exists) {                            console.log('exists:', exists);                            // => true                   });                   path.exists('/does_not_exist', function(exists) {                            console.log('exists:', exists);                            // => false                   });

Note: From Node0.8Version start, existsFrom pathModule moved to fsModule, changed to fs. exists, Except for the namespace, none of the following changes:

                   var fs = require('fs');                   fs.exists('/does_not_exist', function(exists) {                            console.log('exists:', exists);                            // => false                   });

Path. exists () is an I/O operation. Because it is asynchronous, a callback function is required. When an I/O operation returns, this callback function is called, and pass the result to it. You can also use its synchronous version path. existsSync (). The function is the same, but it does not call the callback function, but directly returns the result:

                  var path = require('path');                 path.existsSync('/etc/passwd');                 // => true

Fs module Introduction

The fs module contains functions related to file query and processing. These functions can be used to query file information, read/write, and close files. Import the fs module as follows:

         var fs = require(‘fs’)

Query file information

Sometimes you may need to know the file size, creation date, permission, and other file information. You can use the fs. stath function to query the metadata of a file or directory:

                   var fs = require('fs');                  fs.stat('/etc/passwd', function(err, stats) {                                    if (err) { throw err;}                                    console.log(stats);                  });

This piece of code has output similar to the following:

 { dev: 234881026,ino: 95028917,mode: 33188,nlink: 1,uid: 0,gid: 0,rdev: 0,size: 5086,blksize: 4096,blocks: 0,atime: Fri, 18 Nov 2011 22:44:47 GMT,mtime: Thu, 08 Sep 2011 23:50:04 GMT,ctime: Thu, 08 Sep 2011 23:50:04 GMT }

The fs. stat () call will pass a stats class instance as a parameter to its callback function. You can use the stats instance as follows:

• Stats. isFile () -- returns true if it is a standard file instead of a directory, socket, symbolic link, or device; otherwise, false
• Stats. isDiretory () -- if it is a directory, tue is returned; otherwise, false
• Stats. isBlockDevice () -- returns true if it is a block device. In most UNIX systems, Block devices are usually in the/dev directory.
• Stats. isChracterDevice () -- returns true if it is a character device
• Stats. isSymbolickLink () -- returns true if it is a File Link
• Stats. isFifo () -- returns true if it is a FIFO (a special type of UNIX naming pipeline)
• Stats. isSocket () -- if it is a UNIX socket (TODO: googe it)

Open a file

Before reading or processing files, you must use fs. open function to open the file, and then the provided callback function will be called and get the file descriptor. Later you can use this file descriptor to read and write the opened file:

                   var fs = require('fs');                   fs.open('/path/to/file', 'r', function(err, fd) {                        // got fd file descriptor                   });

Fs. the first parameter of open is the file path, and the second parameter is the mark used to indicate the mode in which the file is opened. The mark can be r, r +, w, w +, a or a +. The following is a description of these tags (from the fopen page of the UNIX document)

• R -- open the file in read-only mode. The initial position of the data stream starts at the file.
• R + -- open the file in read/write mode, and the initial position of the data stream starts at the file
• W -- if the file exists, the file length is cleared to 0, that is, the file content will be lost. If it does not exist, try to create it. The initial location of the data stream starts with the file.
• W + -- open the file in read/write mode. If the file does not exist, create it. If the file exists, clear the file length to 0, that is, the file content will be lost. The initial location of the data stream starts with the file.
• A -- open the file in write-only mode. If the file does not exist, create it. The initial position of the data stream is at the end of the file. Data is appended to the end of the file for each subsequent write operation.
• A + -- open the file in read/write mode. If the file does not exist, create it. The initial position of the data stream is at the end of the file. Data is appended to the end of the file for each subsequent write operation.

Read files

Once the file is opened, you can start to read the file content, but before you start, you have to create a buffer to store the data. This buffer object will be passed to the fs. read function as a parameter and filled with data by fs. read.

var fs = require('fs');fs.open('./my_file.txt', 'r', function opened(err, fd) {if (err) { throw err }var readBuffer = new Buffer(1024),bufferOffset = 0,bufferLength = readBuffer.length,filePosition = 100;fs.read(fd,         readBuffer,         bufferOffset,         bufferLength,         filePosition,         function read(err, readBytes) {                   if (err) { throw err; }                   console.log('just read ' + readBytes + ' bytes');                   if (readBytes > 0) {                            console.log(readBuffer.slice(0, readBytes));                   }});});

The code above tries to open a file. After the file is successfully opened (the opened function is called), the request starts to read the subsequent 100th bytes of data (1024 rows) from the 11th bytes of the file stream ).

The last parameter of fs. read () is a callback function (row 16th). It is called in the following three cases:

• An error occurred.
• Data read successfully
• No data readable

If an error occurs, the first parameter (err) will provide an object containing the error information for the callback function. Otherwise, this parameter is null. If the data is read successfully, the second parameter (readBytes) indicates the size of the data in the read buffer. If the value is 0, it indicates that the data has reached the end of the file.

Note: Once the buffer object is passed to fs. open (), The control of the buffer object is transferred to the readCommand. Only when the callback function is called can the control of the buffer object be returned to you. Therefore, do not read or write data or call other functions to use this buffer object. Otherwise, you may read incomplete data. Worse, you may write data to this buffer object concurrently.

Write files

Pass to fs. write () A buffer object containing data to write data to and from an opened file:

var fs = require('fs');fs.open('./my_file.txt', 'a', function opened(err, fd) {    if (err) { throw err; }    var writeBuffer = new Buffer('writing this string'),    bufferPosition = 0,    bufferLength = writeBuffer.length, filePosition = null;    fs.write( fd,        writeBuffer,        bufferPosition,        bufferLength,        filePosition,        function wrote(err, written) {           if (err) { throw err; }           console.log('wrote ' + written + ' bytes');        });});

In this example, the 2nd line of code attempts to open a file in append mode (a) And then 7th lines of code (the original article is 9) write data to a file. The buffer object must contain several pieces of information as parameters:

• Buffer data
• Where does the data to be written start from the buffer zone?
• Length of data to be written
• Location where the data is written to the file
• The callback function wrote called after the operation is completed

In this example, the filePostion parameter is null, that is, the write function will write the data to the current position of the file pointer. Because the file is opened in append mode, the file pointer is at the end of the file.

Like the read operation, do not use any passed-in buffer object during fs. write execution. Once fs. write starts to execute, it obtains control of the buffer object. You can only use the callback function again after it is called.

Close file

You may have noticed that all examples in this chapter have not closed the file code so far. Because they are only one-time and simple examples, when the Node process ends, the operating system will ensure that all files are closed.

However, in actual applications, once a file is opened, you must ensure that it is eventually disabled. To do this, you need to track all opened file descriptors and call fs. close (fd [, callback]) When you no longer use them to close them. If you are not careful, it is easy to omit a file descriptor. The following example provides a function called openAndWriteToSystemLog, showing how to close the file with caution:

var fs = require('fs');function openAndWriteToSystemLog(writeBuffer, callback){    fs.open('./my_file', 'a', function opened(err, fd) {        if (err) { return callback(err); }        function notifyError(err) {            fs.close(fd, function() {                callback(err);            });        }        var bufferOffset = 0,        bufferLength = writeBuffer.length,        filePosition = null;        fs.write( fd, writeBuffer, bufferOffset, bufferLength, filePosition,            function wrote(err, written) {                if (err) { return notifyError(err); }                fs.close(fd, function() {                    callback(err);                });            }        );    });}openAndWriteToSystemLog(    new Buffer('writing this string'),    function done(err) {        if (err) {            console.log("error while opening and writing:", err.message);            return;        }        console.log('All done with no errors');    });

Here, we provide a function called openAndWriteToSystemLog, which accepts a buffer object containing the data to be written and a callback function called after an operation is completed or error occurs. If an error occurs, the first parameter of the callback function contains the error object.

Note that the internal function policyerror will close the file and report the error.

Note: so far, you know how to use the underlying atomic operations to open, read, write, and close files. HoweverThere is also a set of more advanced constructors that allow you to process files in a simpler way.

For example, you want to use a safe method to allow two or more writeYou can use WriteStream to concurrently append data to a file..

Also, if you want to read a certain area of a file, you can consider using ReadStream. These two kinds of regular meetings are introduced in chapter 9 "Data Reading and Writing.

Summary

When you use a file, you need to process and extract the file path information in most cases. By using the path module, you can connect to the path, standardize the path, and calculate the path difference, and convert relative paths to absolute paths. You can extract path components such as the extension, file name, and directory of a specified file path.

Node provides a set of underlying APIs in the fs module to access the file system. The underlying APIs use file descriptors to operate files. You can use fs. open to open a file, use fs. write to write a file, use fs. read to read the file, and use fs. close to close the file.

When an error occurs, you should always use the correct error processing logic to close the file-to ensure that the opened file descriptors are closed before the call returns.