Hadoop Learning Note 01--hadoop Distributed File system

Hadoop has a distributed system called HDFS , all known as Hadoop distributed Filesystem.

HDFs has a block concept, and the default is that the file on 64mb,hdfs is divided into chunks of block size, as separate storage units. The advantage of using blocks is: 1. A file size can be larger than the capacity of any disk in the cluster network, and all blocks of the file do not need to be stored on the same disk; 2. Simplify storage management; 3. Provide data fault tolerance and availability

The HDFs cluster has 2 types of nodes: one namenode(manager) and multiple datanode( workers). Namenode manages the namespace of the filesystem, maintains the file system tree and all files and directories in the entire tree, which are permanently stored on the local disk as two files: the namespace image file and the edit log file. Datanode is the working node of the file system, storing and retrieving the database as needed, and periodically sending the Namenode a list of the blocks they store.

Hadoop Learning Note 01--hadoop Distributed File system