Hadoop HDFS File System

1. NameNode metadata node: Manage the file system secondarynamenode slave metadata node: Metadata node usage Node 2, DataNode data node: data storage location 1) the client requests to read or write files, metadata node initiation 2) Periodic metadata node retrieval of fast data currently stored 3. Block data blocks

1. NameNode metadata node: Manage the file system secondary namenode slave metadata node: Metadata node usage Node 2, DataNode data node: data storage location 1) the client requests to read or write files, metadata node initiation 2) Periodic metadata node retrieval of fast data currently stored 3. Block data blocks

1. NameNode metadata node: Manage the File System

Secondary namenode slave metadata node: slave metadata Node

2. DataNode data node: data storage location

1) The client requests to read or write files, and the metadata node initiates

2) Periodically think about metadata nodes to return the fast information of the currently stored data

3. Block data blocks: the most basic storage unit. The default value is 64 mb. When the size of a file is smaller than the size of a data Block, it does not occupy the space of the entire data Block.

Write
1), the Client initiates a file write request to the NameNode.
2) according to the file size and file block configuration, NameNode returns the information of the DataNode managed by the Client.
30. The Client divides the file into multiple blocks and writes them to each DataNode Block in sequence based on the DataNode address information.

Read
1), the Client initiates a File Read Request to the NameNode.
2). NameNode returns the DataNode information stored in the file.
3). The Client reads the file information.

Simple operation:

Basic information of the current HDFS
$: Hadoop dfsadmin-report

Configured Capacity: 15217328128 (14.17 GB)
Present Capacity: 8593608704 (8 GB)
DFS Remaining: 8593297408 (8 GB)
DFS Used: 311296 (304 KB)
DFS Used %: 0%
Under replicated blocks: 1
Blocks with primary upt replicas: 0
Missing blocks: 0

-------------------------------------------------
Datanodes available: 1 (1 total, 0 dead)

Name: 127.0.0.1: 50010
Decommission Status: Normal
Configured Capacity: 15217328128 (14.17 GB)
DFS Used: 311296 (304 KB)
Non DFS Used: 6623719424 (6.17 GB)
DFS Remaining: 8593297408 (8 GB)
DFS Used %: 0%
DFS Remaining %: 56.47%
Last contact: Tue Dec 11 01:16:30 CST 2012

List HDFS files

$; Hadoop fs-ls

Found 2 items
-Rw-r -- 3 sina supergroup 13 2012-12-05 0:43/user/demo.txt
Drwxr-xr-x-sina supergroup 0/user/docs

Cat;

$ Hadoop fs-cat/user/demo.txt
Test-test