Hadoop Partitioner and Custom Partitioner

First, Hadoop partitioner

All partitioner inherit from the abstract class Partitioner, implement Getpartition (KEY var1, VALUE var2, intvar3), and the partitioner with Hadoop comes with:

(1) Totalorderpartitioner

Generally used when doing global sorting

(2) Keyfieldbasedpartitioner

(3) Binarypartitioner

public int getpartition (binarycomparable key, V value, int numpartitions) {
    int length = Key.getlength ();
    int Leftindex = (this.leftoffset + length)% length;
    int Rightindex = (this.rightoffset + length)% length;
    int hash = writablecomparator.hashbytes (Key.getbytes (), Leftindex, Rightindex-leftindex + 1);
    Return (hash & 2147483647)% Numpartitions;
  }

(4) Hashpartitioner default partition, determine partition by Ke's hash value

public int getpartition (K key, V value, int numreducetasks) {
    return (Key.hashcode () & 2147483647)% Numreducetask s;
  }

Second, custom Paritioner

Key is ID, implementation is partitioned by the last three bits of key

public class Rtfpartitioner extends Partitioner<text, text> {
  @Override public
  int getpartition (Text key, Text value, int numpartitions) {
    String k = key.tostring ();
    Int par = Integer.parseint (k.substring (K.length ()-2));
    Return par;
  }

}