HDFS Instruction (ii) Movefromlocal,movetolocal,tail,rm,expunge,chown,chgrp,setrep,du,df_hadoop

Objective

This article mainly learn Hadoop HDFs from HDFs move to local, move from local to Hdfs,tail view last, rm delete file, expunge empty trash,chown change owner, setrep change file copy number, CHGRP change belong group,, Du, DF Disk Footprint

Movefromlocal

Copy a local file to HDFs, and when successful, delete the local file
Instruction usage

Movefromlocal <localSrc> <dest>

Example

HDFs Dfs-movefromlocal/home/dataflair/desktop/sample/user/dataflair/dir1

Movetolocal

Similar to-get, but when replication is complete, files on HDFs are deleted
Instruction usage

Movetolocal <src> <localDest>

Example

HDFs Dfs-movetolocal/user/dataflair/dir2/sample/user/dataflair/desktop

Tail

Similar to the tail in Linux, the final 1kb of the file is printed to console or stdout.
Instruction usage

HDFs Dfs-tail [F] <filename>

Example

HDFs dfs-tail/user/dataflair/dir2/purchases.txt
HDFs dfs-tail-f/user/dataflair/dir2/purchases.txt

Rm

Removing files or emptying data under a path
Instruction usage

RM <path>

Example

HDFs dfs-rm/user/dataflair/dir2/sample

Recursive deletion

HDFs Dfs-rm-r/user/dataflair/dir2

Expunge

Empty Trash
The data deletion in the HDFs is also more characteristic, is not deletes directly, but first puts in a similar recycle Bin place (/trash), may restore
For files that the user or application wants to delete, HDFs will rename it and move it to/trash, and when it is overwritten or to a certain lifecycle (now defaults to 6 hours), HDFs is removed from the file, and only this time, Datanode related disk space can save
The expunge class clears the/trash, similar to emptying the Recycle Bin
Instruction usage

HDFs Dfs-expunge

Example

HDFs Dfs-expunge

Chown

Similar to Linux chown, users should be superuser in order to do a second operation
Change the owner of a file
Instruction usage

HDFs Dfs-chown [R] [Owner][:[group]] uri [URI]

Example

HDFs Dfs-chown-r Dataflair/opt/hadoop/logs

Chgrp

Similar to Linux chgrp, users should be superuser in order to do a second operation
Change all groups of files
Instruction usage

HDFs Dfs-chgrp [R] <NewGroupName> <file or directory name>

Example

HDFs Dfs-chgrp [-r] New Group sample

Setrep

Instruction usage
Use to change the number of copies of a file, if it is a folder, then the command will change the number of copies for all files under that folder
-W indicates how much of the replica is changed
Use the-r option to change the number of copies of all directories + files recursively in a directory

Setrep [-R] [-W] Rep <path>

Example

HDFs dfs-setrep-w 3/user/dataflair/dir1

Du

Similar to the Du in Linux, statistics the size of each file in a directory, you can add-h to improve file readability
Instruction usage

Du <path>

Example

HDFs dfs-du/user/dataflair/dir1/sample

Df

Similar to the DU in Linux, query the size of a directory, you can add-h to improve the readability of the file
Instruction usage

HDFs DFS-DF [-h] uri [URI ...]

Example

HDFs dfs-df-h

Translation of the original: http://data-flair.training/blogs/most-used-hdfs-commands-tutorial-examples/