Hadoop Webhdfs Setup and usage instructions

1. Configuration

Namenode Hdfs-site.xml is the Dfs.webhdfs.enabled property must be set to true, otherwise you will not be able to use Webhdfs liststatus, listfilestatus, etc. need to list files, The folder Status command, because this information is saved by Namenode.

Add attributes to/etc/hadoop/conf/hdfs-site.xml in Namenode and one datanode:

Double-click code Select All

1 2 3 4

< property > < name >dfs.webhdfs.enabled</name > < value >true</value > </property >

2. Instructions for use

Access Namenode HDFs using port 50070, Access Datanode Webhdfs using 50075 ports. Access files, folder information using Namenode IP and 50070 ports, access to file content or open, upload, modify, download and other operations using Datanode IP and 50075 ports. To do all of the WEBHDFS operations using Namenode's IP and port without differentiating the ports, you need to set dfs.webhdfs.enabled in Hefs-site.xml to true on all datanode.

3.curl Operation Webhdfs

Create and write a file
Curl-i-X PUT "Http://localhost:50070/webhdfs/v1/<PATH>?op=CREATE
[&overwrite=<true|false>] [&blocksize=<long>] [&replication=<short>]
[&permission=<octal>] [&buffersize=<int>] "
Curl-i-X put-t <LOCAL_FILE> "HTTP://<DATANODE>:<PORT>/WEBHDFS/V1/<PATH>?
Op=create ... "
Note that this place is filled with datanode information.

Append content within a file
Curl-i-X POST "HTTP://<HOST>:<PORT>/WEBHDFS/V1/<PATH>?OP=APPEND[&BUFFERSIZE=<INT>]"
Curl-i-X post-t <LOCAL_FILE> "HTTP://<DATANODE>:<PORT>/WEBHDFS/V1/<PATH>?
Op=append ... "
Note that the command obtains the Datanode information.

Open and read a file
Curl-i-L "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=OPEN
[&offset=<long>] [&length=<long>] [&buffersize=<int>] "

Create a Directory
Curl-i-X PUT "HTTP://<HOST>:<PORT>/<PATH>?OP=MKDIRS[&PERMISSION=<OCTAL>]"

Rename files, folders
Curl-i-X PUT "<HOST>:<PORT>/webhdfs/v1/<PATH>?op=RENAME&destination=<PATH>"

Delete files/folders
Curl-i-X DELETE "Http://

status information for files/folders
Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETFILESTATUS"

Directory list
Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=LISTSTATUS"

Get context rollup information for a directory
Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETCONTENTSUMMARY"

Get the check Sum File
Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETFILECHECKSUM"

Get home Directory
Curl-i "Http://<HOST>:<PORT>/webhdfs/v1/?op=GETHOMEDIRECTORY"

Set permissions
Curl-i-X PUT "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=SETPERMISSION [&permission=< Octal>] "

Set owner
Curl-i-X PUT "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=SETOWNER [&owner=<user>][ &group=<group>] "

Set up Backup
Curl-i-X PUT "Http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=SETREPLICATION [&replication=< Short>] "