The shell command for HDFs

One. HDFs shell command

We all know that HDFs is a distributed file system to access data, then the operation of HDFs is the basic operation of the file system, such as file creation, modification, deletion, modify permissions, folder creation, deletion, renaming and so on. The operation of the HDFs command is similar to the Llinux Shell's operation of the file, such as LS, mkdir, RM, and so on.
When we perform HDFS shell operations, make sure Hadoop is working, and we can use the JPS command to ensure that each Hadoop process is seen.
We can execute commands for Hadoop fs to view all the commands for HDFS shell operations as follows.
[[email protected] ~]# Hadoop FS
Usage:java Fsshell
[-ls <path>]
[-LSR <path>]
[-du <path>]
[-dus <path>]
[-count[-q] <path>]
[-MV <src> <dst>]
[-CP <src> <dst>]
[-RM [-skiptrash] <path>]
[-RMR [-skiptrash] <path>]
[-expunge]
[-put <localsrc> ... <dst>]
[-copyfromlocal <localsrc> ... <dst>]
[-movefromlocal <localsrc> ... <dst>]
[-get [-IGNORECRC] [-CRC] <src> <localdst>]
[-getmerge <src> <localdst> [ADDNL]]
[-cat <src>]
[-text <src>]
[-copytolocal [-IGNORECRC] [-CRC] <src> <localdst>]
[-movetolocal [-CRC] <src> <localdst>]
[-mkdir <path>]
[-setrep [-R] [-W] <rep> <path/file>]
[-touchz <path>]
[-test-[ezd] <path>]
[-STAT [format] <path>]
[-tail [-F] <file>]
[-chmod [-R] <mode[,mode] ... | Octalmode> PATH ...]
[-chown [-R] [Owner][:[group]] PATH ...]
[-CHGRP [-R] GROUP PATH ...]
[-help [CMD]]

Two. The shell operation command options for HDFs

Option name	Use formatting	Meaning
-ls	-ls < paths >	View the current directory structure for the specified path
-lsr	-LSR < paths >	Recursively view the directory structure of a specified path
-du	-du < paths >	The size of each file in the statistics directory
-dus	-dus < paths >	Size of file (clip) in summary statistics directory
-count	-count [-Q] < paths >	Number of statistical files (clips)
-mv	-MV < Source path > < destination path >	Move
-cp	-CP < Source path > < destination path >	Copy
-rm	-RM [-skiptrash] < paths >	Delete files/blank folders
-rmr	-RMR [-skiptrash] < paths >	Recursive deletion
-put	-put < files on multiple Linux >	Uploading files
-copyfromlocal	-copyfromlocal < files on multiple Linux >	Copy from local
-movefromlocal	-movefromlocal < files on multiple Linux >	Move from local
-getmerge	-getmerge < Source path > <linux path >	Merge to Local
-cat	-cat	View File Contents
-text	-text	View File Contents
-copytolocal	-copytolocal [-IGNORECRC] [-CRC] [HDFs source path] [Linux destination Path]	Copy to Local
-movetolocal	-movetolocal [-CRC]	Move to Local
-setrep	-setrep [-R] [-W] < number of copies > < paths >	Modify the number of replicas
-mkdir	-mkdir	Create a blank folder
-touchz	-touchz < file paths >	Create a blank file
-stat	-stat [format] < paths >	displaying file statistics
-tail	-tail [-F] < files >	View File trailer Information
-chmod	-chmod [-R] < permission mode > [path]	Modify Permissions
-chown	-chown [-R] [genus Master][:[]] path	Modify Owner
-chgrp	-CHGRP [-R] Genus group name path	Modify Genus Group
-help	-help-help [Command Options]	Help

Iii. usage of each command option 1. LS Displays the current directory structure

(1) the command option indicates that the current directory structure of the specified path is viewed, followed by the HDFs path, if shown

Let's explain the content format for each line:

① The initial letter indicates the folder (if "D") or the file (if "-");
The 9-bit character following the ② represents the permission;
The number after the ③ or "-" indicates how many copies. If it is a file, use a number to represent the number of copies;
The "root" in the back of ④ is the owner;
The "supergroup" in the back of ⑤ represents a group;
The "0", "84927175" after ⑥ indicates the file size, in bytes;
The time after the ⑦ represents the modified time, the format is month and day hours;
⑧ The last entry represents a file path.
The visible root directory has 1 folders and 1 files.

(2) If there is no path after the command option, the/user/< Current user > directory is accessed. We use the root user to log in, so we will access the/user/root directory of HDFs, but if there is no such directory/user/root, there will be a hint that the file does not exist as in error 3.1, after adding the directory, execute the command 3.2, 3.3, as shown.

Figure 3.1

Figure 3.2

Figure 3.3

2.-LSR Recursive display directory structure

This command option indicates that the directory structure of the current path is recursively displayed, followed by the HDFs path. As shown in.

There is a root directory under the/user directory, and the root directory has a file Hello

3.-du the file size under the statistics directory

The command option displays the file size under the specified path, in bytes, as shown in

4.-dus the file size in the summary statistics directory

The command option displays the file size of the specified path, in bytes, as shown in

5. Number of Count statistics files (clips)

This command option displays the number of folders, the number of files, and the total file size information under the specified path, as shown in

6. MV Mobile

This command option represents moving HDFs files into the specified HDFs directory. followed by two paths, the first represents the source file, and the second represents the destination directory. As shown

There are three commands in order to reflect the changes before and after the move.

7. CP Replication

This command option indicates that the file specified by the HDFs is copied into the specified HDFs directory. followed by two paths, the first one is the copied file, the second is the destination. As shown

There are three commands in order to reflect the changes before and after replication.

8. RM Delete File/Blank folder

The command option indicates that the specified file or empty directory is deleted, as shown in

In, the first three orders are to reflect the changes before and after the implementation of the situation. The fourth command is to delete the non-empty "/user" directory, the operation failed, indicating that the non-empty directory cannot be deleted

9. RMR Recursive deletion

This command option indicates that all subdirectories and files under the specified directory are deleted recursively, as shown in

Ten. Put upload file

This command option means that the files on Linux are copied to HDFs as shown in

Copyfromlocal copy from local to HDFs

The operation is consistent with-put, no longer an example.

Movefromlocal moving from local to HDFs

This command indicates that the file is moved from Linux to HDFs as shown in

Getmerge Merge to Local

The meaning of this command option is to merge all the file contents of the HDFs specified directory into a local Linux file, as shown in

Cat View File Contents

The command option is to view the contents of the file as shown in

. text View File contents

The command option can be thought of as acting and using the same as-cat, here slightly.

mkdir Creating a blank folder

The command option indicates that the folder is created, followed by the folder in which HDFs will be created, as shown in

Setrep. Set Number of replicas

(1) the command option is to modify the number of copies of the saved file, followed by the number of copies, followed by the file path, as shown in

In Figure 3.17, we modified the number of copies of the file/file1, changed from 1 to 2, meaning that one more copy, HDFS will automatically perform the file copy work, generating a new copy.

(2) if the last path represents a folder, then the option-r is required to modify the copy for all files in the folder, as shown in

In, we have the/user/root folder operation, using the option-R, then the/user/root file file2,file1 copy number has changed.

(3) Another option is-W, which indicates that the pending copy operation ends before exiting the command, as shown in

Touchz Creating a blank file

The command option is to create a blank file in HDFs, as shown in

Statistical information for the stat display file

This command option displays some statistics about the file, as shown in

In, the command options can be formatted with quotation marks. The format "%b%n%o%r%Y" In the example indicates file size, file name, block size, number of copies, Access time

Tail viewing the contents of a file trailer

This command option displays the contents of the last 1 K bytes of the file. Typically used to view logs. If you have option-F, the file contents will be displayed automatically when the content changes. As shown

chmod. Modify file Permissions

(1) the use of this command option is similar to the chmod usage in the Linux shell, which is to modify the permissions of the file as shown in

3

(2) in Figure 3.23, the permissions of the file/emptyfile are modified. If you add the option-r, you can modify the permissions for all files in the folder, as shown in

Chown Modification of the owner

The command option represents the owner of the modified file, as shown in

(2) the owner of the file/emptyfile is changed from root to Sunddenly. You can also modify the genus Group, as shown in

In, the file/emptyfile and the owner of the group are modified to Itcast, if only to modify the genus group, you can use ": sunddenly". If you have the option-r, it means that you can recursively modify the owner and group information of all the files in the folder

Chgrp modified Genus Group

The purpose of this command is to modify the genus Group of the file, which is equivalent to the usage of "Chown: Genus", as shown in

Help

The command option displays the Help information, followed by the command options that need to be queried, as shown in

The shell command for HDFs