Shell command interface of the storage system based on key/value + hadoop HDFS

Parse all commands in hadoop HDFS (the operation process is your own idea. You are welcome to correct them)

Interface Name	Function	Procedure
Get	Copy the file to the local file system. If multiple source files are specified, the local destination must be a directory.	(1) According to the above mechanism, the storage engine on the config server reads the K-V layer by layer until the file name (or large file metadata information) is obtained ); (2) obtain the corresponding file from the corresponding data server based on the file name (or big file metadata) (large files need to be spliced ); (3) write the obtained file to the local file system.
Put	Copy a file or directory from the local file system to the target file system.	(1) parse the <DST> absolute path layer by layer to obtain the key/value data of the target folder, as described in the preceding chapter; (2) generate the corresponding key/value based on the rules described in the chapter and the labels of the target folder and the file name of the source file; (3) Upload the relevant information of the source file based on the key/value generated in step (2; (4) Upload the source file content to the system.
Rm	Delete a specified file. Only non-empty directories and files are deleted.	(1) as described in the preceding chapter, the key/value of the specified file is obtained from the location where the target file is located layer by layer; (2) Find the information stored on the datanode where the specified file or directory is located based on the key; (3) Delete information; (4) write the deletion record to the edit log.
Mkdir	Create a directory to create all the missing parent directories in the path.	(1) According to the above mechanism, find the parent directory layer by layer and obtain the tag. If the parent directory does not exist, recursively create the directory on the path; (2) mark the parent directory as the parent node field of the key and create a new directory.
Appendtofile	Add the source file from the local file system to the target file system.	(1) parse the <DST> absolute path layer by layer to obtain the key/value data of the target folder, as described in the preceding chapter; (2) generate the corresponding key/value based on the rules described in the chapter and the labels of the target folder and the file name of the source file; (3) Upload the relevant information of the source file based on the key/value generated in step (2; (4) Upload the source file content to the system.
Cat	Output the content of the specified file in the path to stdout.	(1) According to the above mechanism, the path is parsed to the target file layer by layer to obtain the key/value of the file; (2) output the file content to stdout based on the key value; (3) display the result on the terminal.
Chgrp	Groups that change files and directories.	(1) According to the above mechanism, the storage engine on the config server reads the K-V layer by layer until the operated file name/directory (or large file metadata information) is obtained ); (2) modify the field that represents the group of value in this K-V and re-insert the new K-v.
Chmod	Change the access permissions of files and directories.	(1) According to the above mechanism, the storage engine on the config server reads the K-V layer by layer until the operated file name/directory (or large file metadata information) is obtained ); (2) modify the value field in this K-V to indicate access permissions and re-insert the new K-v.
Chown	Change the owner of the file and directory.	(1) According to the above mechanism, the storage engine on the config server reads the K-V layer by layer until the operated file name/directory (or large file metadata information) is obtained ); (2) modify the field of value in this K-V that represents the owner and re-insert the new K-v.
Copyfromlocal	Copy a file from a local file system, similar to the PUT command.	For the operation procedure, see put interface.
Coptolocal	Copy the file to the local file system, which is similar to the GET command.	For the operation procedure, see get interface.
Count	Displays the number of subdirectories, the number of files, the number of bytes used, and the names of all file directories.	(1) Parse to the target path and obtain the relevant key/value; (2) obtain the number of sub-directories, the number of files, and fields related to information such as strings.
CP	Copy the file from the Source Path to the target path. This command allows multiple original paths. The target path must be a directory.	(1) Parse <DEST> layer by layer to obtain the location information of the target file, as described in the preceding chapter; (2) obtain the content of the source file; (3) copy the source file information to the target file one by one.
Du	The file size is displayed. If path is a directory, the size of all files in the directory is displayed.	(1) According to the above mechanism, the storage engine on the config server reads the K-V layer by layer until the file name or directory is obtained; (2) If the file reads the corresponding key/value, read the field indicating the file size in the value, and read the mark if it is a directory, recursively query all files in this directory.
DUS	Displays the file size, but when used as a directory, the total file size is displayed.	(1) According to the above mechanism, the storage engine on the config server reads the K-V layer by layer until the file name or directory is obtained; (2) If the file reads the corresponding key/value, read the field indicating the file size in the value, and read the mark if it is a directory, recursively query the value information of all files under this directory, and sum the value field information about the file size to sum up and.
Expunge	Clear the recycle bin.	(1) According to the above mechanism, parse it to the HDFS system recycle bin directory (/user name/. Trash/current ); (2) Delete (refer to interface RM ).
Getfacl	Displays the access control list of files and directories.	(1) obtain the key/value data of the target file or path through layer-by-layer parsing as described in the preceding chapter; (2) obtain information about ACL fields in value.
Getmerge	Obtain all the files specified by the source directory, merge them into a single file, and write them to localdst in the local file system.	(1) perform layer-by-layer parsing based on the above mechanism to obtain the mark of the source directory file; (2) Search for the sub-files under the directory file in config Server Based on the tag; (3) create a target file and merge the contents of the sub-file into the target file in sequence; (4) write the target file to the local file system.
Ls	List Directory and file information.	(1) According to the above mechanism, the tag of the target file or folder is obtained through layer-by-layer parsing; (2) query sub-directories or file-related information based on tags; (3) list information.
LSR	List directories, subdirectories, and file information cyclically.	For the operation procedure, refer to the LS interface.
Movefromlocal	Similar to put, the local source will be deleted after being copied.	(1) parse the <DST> absolute path layer by layer to obtain the key/value data of the target folder, as described in the preceding chapter; (2) generate the corresponding key/value based on the rules described in the chapter and the labels of the target folder and the file name of the source file; (3) Upload the relevant information of the source file based on the key/value generated in step (2; (4) Upload the source file content to the system; (5) Delete the local source file.
Movetolocal	Similar to the GET command.	For the operation procedure, see get interface.
MV	Move the file from the source address to the target address.	(1) perform layer-by-layer parsing of the <DEST> absolute path to obtain the label of the target folder; (2) parse the source file layer by layer to obtain the key/value data of the source file; (3) change the key value in the source file and change the parent node label field to the label of the target folder.
RMR	Recursively deletes specified files and directories.	(1) According to the above mechanism, the tag of the specified file is obtained by parsing the target file location layer by layer; (2) Use recursive algorithms to delete specified files or directories Based on tags.
Setfacl	Sets the files and directories of the access control list (ACL.	(1) perform layer-by-layer parsing to obtain the key/value data of the target file or path; (2) set the ACL in the value.
Setrep	Change the target copy coefficient of the file.	(1) perform layer-by-layer parsing to obtain the key/value data of the target file as described in the preceding chapter; (2) modify the copy coefficient field in value.
Stat	Displays statistics in the file.	(1) perform layer-by-layer parsing to obtain the key/value data of the target file as described in the preceding chapter; (2) obtain the fields related to file statistics in value.
Tail	Displays the last 1 kb of data in the target file.	(1) perform layer-by-layer parsing to obtain the key/value data of the target file as described in the preceding chapter; (2) query the 1kb information in the last part based on the obtained key/value. If the last part is less than 1kb, query the information in the last part and splice it into 1kb. Return information.
Test	Check the path.	(1) parse the target path layer by layer to obtain the key/value data of the target file as described in the previous chapter; (2) Check the key/value fields in the target file to check whether the file exists, whether the size is 0, and whether the directory path exists.
Text	Displays the text of a file. When the file is a text file, it is equivalent to Cat. When the file is in the compressed format, it is extracted first.	For the operation procedure, see CAT interface.
Touchz	Create a file with a length of 0.	For the operation procedure, see the mkdir interface.