HDFs Replica Selection Policy

Before the client writes to the Datanode block, it communicates with the Namenode, and the namenode selects the specified number of Datanode to hold the copy. The specific replica selection policy is in the Blockplacementpolicy interface, and its subclass implementation is blockplacementpolicydefault. There will be more than one choosetarget () method overload in the class, but the following method is eventually called:

1 /**2 * This is the "not" part of the "public API" but was used by the unit tests.3    */4Datanodedescriptor[] Choosetarget (intNumofreplicas,5 datanodedescriptor Writer,6List<datanodedescriptor>Chosennodes,7Hashmap<node, node>Excludednodes,8                                     Longblocksize) {9       //Numofreplicas: Number of replicas to selectTen       //clustermap.getnumofleaves (): The number of DN for the entire cluster One     if(Numofreplicas = = 0 | | clustermap.getnumofleaves () ==0) { A       return NewDatanodedescriptor[0]; -     } -        the     //excludednodes: Excluded DN (because some of the DN is already selected, so they are no longer selected) -     if(Excludednodes = =NULL) { -Excludednodes =NewHashmap<node, node>(); -     } +       -     intClustersize =clustermap.getnumofleaves (); +     //total number of replicas = number selected + number of replicas specified A     intTotalnumofreplicas = chosennodes.size () +Numofreplicas; at     if(Totalnumofreplicas > Clustersize) {//if the total number of replicas > The DN of the entire cluster -Numofreplicas-= (totalnumofreplicas-clustersize); -Totalnumofreplicas =clustersize; -     } -        -     //calculates how many DN of each rack can be selected in     intMaxnodesperrack = -(totalNumOfReplicas-1)/clustermap.getnumofracks () +2; to        +list<datanodedescriptor> results = -       NewArraylist<datanodedescriptor>(chosennodes); the      for(Datanodedescriptor node:chosennodes) { *       //add localMachine and related nodes to Excludednodes $ addtoexcludednodes (node, excludednodes);Panax Notoginseng adjustexcludednodes (excludednodes, node); -     } the        +     //client is not DN A     if(!Clustermap.contains (writer)) { theWriter=NULL; +     } -        $     BooleanAvoidstalenodes = (Stats! =NULL&&Stats $ . Shouldavoidstaledatanodesforwrite ()); -      -     //Select Numofreplicas DN and return the local DN theDatanodedescriptor Localnode =Choosetarget (Numofreplicas, writer, - excludednodes, BlockSize, Maxnodesperrack, results, avoidstalenodes);Wuyi        the Results.removeall (chosennodes); -        Wu     //sorting nodes to form a pipeline -     //organizes the selected DN (the element in result) into a pipe About     returnGetpipeline ((writer==NULL)?Localnode:writer, $Results.toarray (Newdatanodedescriptor[results.size ()]); -}

The meaning of the method is probably the one written in the note, but note the meaning of the variable. In line 48th, the Choosetarget () method is called to select the specified number of DN (the selected DN is stored in result) and returns a DN as the local DN. This method is analyzed below.

HDFs Replica Selection Policy