Hadoop Webhdfs Settings and usage instructions

1. Configure

The hdfs-site.xml of Namenode is that you must set the Dfs.webhdfs.enabled property to True, otherwise you cannot use the Webhdfs liststatus, listfilestatus, etc. need to list files, The folder Status command, because the information is saved by Namenode.

Add attributes to/etc/hadoop/conf/hdfs-site.xml in Namenode and one datanode:

<property>  
<name>dfs.webhdfs.enabled</name>  
<value>true</value>  
< /property>

2. Instructions for use

Access to Namenode's HDFs uses port 50070, and access to Datanode's Webhdfs uses port 50075. Access to file, folder information using Namenode IP and Port 50070, access to file content or to open, upload, modify, download and other operations using Datanode IP and Port 50075. To do all the WEBHDFS operations directly using Namenode IP and port without distinguishing the ports, you need to set the dfs.webhdfs.enabled on all Datanode hefs-site.xml to True.

3.curl Operation Webhdfs

Create and write a file

Curl-i-X put "Http://localhost:50070/webhdfs/v1/<PATH>?op=CREATE

[&overwrite=<true|false>] [&blocksize=<long>] [&replication=<short>]

[&permission=<octal>] [&buffersize=<int>] "

Curl-i-X put-t <LOCAL_FILE> "HTTP://<DATANODE>:<PORT>/WEBHDFS/V1/<PATH>?

Op=create ... "

Note that this place is filled with datanode information.

Append content to a file

Curl-i-X POST "HTTP://<HOST>:<PORT>/WEBHDFS/V1/<PATH>?OP=APPEND[&BUFFERSIZE=<INT>]"

Curl-i-X post-t <LOCAL_FILE> "HTTP://<DATANODE>:<PORT>/WEBHDFS/V1/<PATH>?

Op=append ... "

Note that this command obtains datanode information.

Open and read a file

Curl-i-L "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=OPEN

[&offset=<long>] [&length=<long>] [&buffersize=<int>] "

Create a Directory

Curl-i-X put "HTTP://<HOST>:<PORT>/<PATH>?OP=MKDIRS[&PERMISSION=<OCTAL>]"

Duplicate files, folders

Curl-i-X put "<HOST>:<PORT>/webhdfs/v1/<PATH>?op=RENAME&destination=<PATH>"

Delete files/folders

Curl-i-X DELETE "Http://

status information for File/folder

Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETFILESTATUS"

Directory listings

Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=LISTSTATUS"

Get context rollup information for a directory

Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETCONTENTSUMMARY"

Get Check Sum File

Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETFILECHECKSUM"

Get home Directory

Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/?op=GETHOMEDIRECTORY"

Set permissions

Curl-i-X put "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=SETPERMISSION [&permission=< Octal>] "

Set owner

Curl-i-X put "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=SETOWNER [&owner=<user>][ &group=<group>] "

Set up a backup

Curl-i-X put "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=SETREPLICATION [&replication=< Short>] "

Author: csdn ROOTSONGJC

See more highlights of this column: http://www.bianceng.cnhttp://www.bianceng.cn/webkf/tools/