Hadoop讀書筆記（二）HDFS的shell操作

標籤：hadoop   hdfs   shell操作   

Hadoop讀書筆記（一）Hadoop介紹：http://blog.csdn.net/caicongyang/article/details/39898629

1.shell操作

1.1所有的HDFS shell操作命名可以通過hadoop fs擷取：

[[email protected] ~]# hadoop fs
Usage: java FsShell
           [-ls <path>]
           [-lsr <path>]
           [-du <path>]
           [-dus <path>]
           [-count[-q] <path>]
           [-mv <src> <dst>]
           [-cp <src> <dst>]
           [-rm [-skipTrash] <path>]
           [-rmr [-skipTrash] <path>]
           [-expunge]
           [-put <localsrc> ... <dst>]
           [-copyFromLocal <localsrc> ... <dst>]
           [-moveFromLocal <localsrc> ... <dst>]
           [-get [-ignoreCrc] [-crc] <src> <localdst>]
           [-getmerge <src> <localdst> [addnl]]
           [-cat <src>]
           [-text <src>]
           [-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>]
           [-moveToLocal [-crc] <src> <localdst>]
           [-mkdir <path>]
           [-setrep [-R] [-w] <rep> <path/file>]
           [-touchz <path>]
           [-test -[ezd] <path>]
           [-stat [format] <path>]
           [-tail [-f] <file>]
           [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
           [-chown [-R] [OWNER][:[GROUP]] PATH...]
           [-chgrp [-R] GROUP PATH...]
           [-help [cmd]]

Generic options supported are
-conf <configuration file>     specify an application configuration file
-D <property=value>            use value for given property
-fs <local|namenode:port>      specify a namenode
-jt <local|jobtracker:port>    specify a job tracker
-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.


The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]

1.2常見操作

所有的HDFS操作均以hadoop fs開頭加上相對應的操作

1.2.1列出HDFS檔案下面所有的檔案

[[email protected] ~]# hadoop fs -ls hdfs://hadoop:9000/

hdfs://hadoop:9000 為hadoop設定檔core-site.xml中配置的預設的檔案系統名稱，上述命令可以簡寫為：

[[email protected] ~]# hadoop fs -ls /

1.2.1檔案上傳：講Llinux下的/usr/local/hadoop-1.1.2.tar.gz上傳到hdfs下的/download檔案夾下

[[email protected] ~]# hadoop fs -ls /usr/local/hadoop-1.1.2.tar.gz  /download

1.2.2查看上傳的檔案：迴圈列出/download下面的所有檔案

[[email protected] ~]# hadoop fs -lsr  /download

1.3HDFS shell操作命令協助

[[email protected] ~]# hadoop fs -help chown

-chown [-R] [OWNER][:[GROUP]] PATH...
                Changes owner and group of a file.
                This is similar to shell‘s chown with a few exceptions.

        -R      modifies the files recursively. This is the only option
                currently supported.

                If only owner or group is specified then only owner or
                group is modified.

                The owner and group names may only cosists of digits, alphabet,
                and any of ‘[email protected]/‘ i.e. [[email protected]/a-zA-Z0-9]. The names are case
                sensitive.

                WARNING: Avoid using ‘.‘ to separate user name and group though
                Linux allows it. If user names have dots in them and you are
                using local file system, you might see surprising results since
                shell command ‘chown‘ is used for local files.

歡迎大家一起討論學習！

有用的自己收!

記錄與分享，讓你我共成長！歡迎查看我的其他部落格；我的部落格地址：http://blog.csdn.net/caicongyang

Hadoop讀書筆記（二）HDFS的shell操作