In HDFs, the fsck command is provided to check the health status of files and directories on HDFs, to obtain block block information and location information for files, and so on.
Specific command description:
-move: Move the corrupted file into the/lost+found directory
-delete: Deleting corrupted files
-openforwrite: The file being written in the output detection
-list-corruptfileblocks: Output corrupted blocks and the files to which they belong
-files: Output files that are being detected
-blocks: Detailed report of output block (need to use with-files parameter)
-locations: Output block location information (need to use with-files parameter)
-racks: The rack information where the output file block is located (needs to be used with the-files parameter)
For example, to view the specific distribution of block blocks for a file in HDFs, you can write:
Hadoop fsck/your_file_path-files-blocks-locations-racks
Specific examples:
#hdfs Fsck/tmp/test/input/word.log-files-blocks-locations-racks connecting to Namenode via Http://c12-138:50070FSCK Started by Root (auth:simple) from/10.254.13.141-Path/tmp/test/input/word.log at Wed 17:17:45 CST 2016/tmp/t Est/input/word.log bytes, 1 block (s): OK0. bp-1390896613-10.254.12.138-1467366342106:blk_1074295976_555178 len=70 live_repl=3 [/default/10.254.13.141:50010, /default/10.254.12.139:50010,/default/10.254.12.138:50010]status:healthy total size:70 B Total dirs:0 Total file S:1 Total symlinks:0 blocks (validated): 1 (avg. block size B) minimally replicated block S:1 (100.0%) over-replicated blocks:0 (0.0) under-replicated blocks:0 (0.0) mis-replicated blocks:0 (0.0) Default replication Factor:3 Average block replication:3.0 corrupt blocks:0 Missing replicas: 0 (0.0%) Number of data-nodes:10 number of RACKS:1FSCK ended at Wed-17:17:45 CST 1 millisecondsthe filesystem under path '/tmp/test/input/word.log ' is HEALTHY
其中:
0. Indicates the number of block;
bp-1390896613-10.254.12.138-1467366342106:blk_1074295976_555178表示block id;
len=
表示该文件块大小;
repl=3 表示该block块的副本数;
Hadoop Command fsck command