The-namenode of the study record of HDFs source code (i)

Namenode maintains the two-tier relationship of the HDFs namespace:

1) The relationship between the file and the data block (inodefile,inodedirectory)

2) The relationship between the data block and the Datanode (Blockmap.blockinfo)

First, put a class diagram:

1. Inode class:

Inode mimics the inode of an index node in a Linux file system. The Inode is an abstract class, the inode holds the filename, the file owner, the file's access rights, the file parent directory (inodedirectory reference, whether it is the Inodefile subclass of the Inode or the Inodedirectory subclass, has the parent directory), Modification time, access time and so on. The key methods in the Inode are listed as follows:

1) RemoveNode, delete the file itself from the parent directory. Call Inodedirectory's RemoveChild method to remove the child file from the parent.

  Boolean RemoveNode () {    ifnull) {      returnfalse;     Else {            Parent.removechild (this);       NULL ;       return true ;    }  }

2) The Isunderconstruction method determines whether the file represented by this inode is in the build (written), the Inode is the parent class by default, and the value returned is false. The Inodeunderconstruction subclass, described below, inherits the Inode and writes the Isunderconstruction method, returning True.

  /** * Is this    inode being constructed?    */  Boolean isunderconstruction () {    returnfalse;  }

2. Inodefile class

Inodefile is a subclass of the inode that represents a specific file. The blocks represented in Inodefile represents a series of blocks of data that comprise a file, where the Blockinfo class inherits from the Block class, which contains not only the inode corresponding to that block, It also contains the Datanode (Datanodedescriptor) in which the block is located. Later, the Blockinfo class will be discussed. The long-type property, saved in the Inodefile class, holds the number of copies of the Block and the recommended block size.

  // Header Mask 64-bit representation   // Format: [Bits for replication][48 bits for preferredblocksize]  Static Final Long Headermask = 0xffffL << blockbits;   protected Long header;   protected null;

The Collectsubtreeblocksandclear method is an abstract method in the Inode abstract class, and the Inodefile class and the Inodedirectory class specifically implement the abstract method. The Collectsubtreeblocksandclear method in the Inodefile class is implemented simply by setting the block array in the inodefile to NULL, At the same time, each block in the deleted blocks is added to the parameters of the Collectsubtreeblocksandclear method.

  int collectsubtreeblocksandclear (list<block> v) {    null;      for (Block blk:blocks) {      v.add (blk);    }     NULL ;     return 1;  }

3. Inodedirectory class

The list of inode is saved in Inodedirectory, and the inode of each sub-file under the directory is saved. And the inodedirectory itself does not contain block.

Private List<inode> children;

The Collectsubtreeblocksandclear method in Inodedirectory is as follows:

 int  collectsubtreeblocksandclear (list<block> V) { int  total = 1 if  (children = = null   return   total;  for  += Child.collectsubtreeblocksandclear (v);    } Parent  = null  ;    Children  = null  ;   return   total; }

The implementation is done by traversing the Inodedirectory children (subdirectory/file), and recursively calling the Inode.collectsubtreeblocksandclear method, deleting subdirectories, directories under files. and counts all cleared subdirectories, the number of child files, and the total variable returned.

4. Inodedirectorywithquota class

The Inodedirectorywithquota class inherits from the Inodedirectory class, and administrators can configure quotas for the directory. You can configure the number of names under the directory, or you can configure the space quotas for all files under subdirectories.

class extends inodedirectory {  privatelong///NameSpace Quota namespace quota, refers to the total number of directory files  Privatelong  nscount;   Private Long // /Disk space quota quota  Private Long diskspace;
  ···}

5. Inodefileunderconstruction class

The Inodefileunderconstruction class inherits from the Inodefile class, which represents the inode in the build state, which becomes a build state when the client opens the file for write or append, and in the HDFs directory tree, The node is a Inodefileunderconstruction object.

classInodefileunderconstructionextendsinodefile {String clientName; //Lease holder Lease Holder  Private FinalString clientmachine;//The host on which the client residesPrivate FinalDatanodedescriptor Clientnode;//If client is a cluster node too. If clients are running on machines in a cluster, they represent data node information  Private intPrimarynodeindex =-1;//The node working on lease recovery  PrivateDatanodedescriptor[] Targets =NULL;//locations for last block Data Flow pipeline member  Private LongLastrecoverytime = 0;//start Time of lease recovery
......}

Leases provide the client with permission to write for a certain period of time. Equivalent to a write lock. ClientName represents the client name that initiates the write operation, which is the client name that holds the lease. Clientmachine the host on which the client resides. Clientnode means that if the client is a node running in a cluster, the variable represents that node. Targets represents the data flow pipeline member for the last block of data. Primarynodeindex and Lastrecoverytime are used for data block recovery (lease recovery).

Inodefileunderconstruction calls Isunderconstruction to determine whether the file is being built.

  /** * Is this    inode being constructed?    */   @Override  boolean  isunderconstruction () {    returnTrue ;  }

The-namenode of the study record of HDFs source code (i)