Hadoop: input, output, key, value format

Map: (K1, V1) → list (K2, V2)
Reduce: (K2, list (V2) → list (K3, V3)

(K1, V1 ):
Jobconf. setinputkeyclass (K1.Class);
Jobconf. setinputvalueclass (v1.Class);

List (K2, V2 ):
Job. setmapoutputkeyclass (K2.Class);
Job. setmapoutputvalueclass (V2.Class);

List (K3, V3 ):
Jobconf. setoutputkeyclass (K3.Class);
Jobconf. setoutputvalueclass (V3.Class);
 

 Jobconf. setinputformat (myinputformat. Class); inputformat: Textinputformat: used to read plain text files. Files are divided into a series of rows ending with LF or Cr. The key is the offset of each row (longwritable), and the value is the content of each row (text ). Keyvaluetextinputformat: used to read files. If the row is split into two parts by a separator, the first part is the key, and the remaining part is the value. If there is no separator, the entire row is used as the key, and the value is null. Sequencefileinputformat: used to read sequencefile. The read format must be consistent with the setoutputkeyclass set when writing sequencefileoutputformat (Key + value format ). Sequencefileinputfilter: Based on the filter, the data that meets the conditions is obtained from sequencefile, and the filter is specified through setfilterclass. Three filters are built in, and the record that regexfilter obtains the key value that meets the specified regular expression; percentfilter uses the specified parameter F to retrieve records with the number of record rows F % = 0; md5filter uses the specified parameter F to retrieve records with MD5 (key) % F = 0. Example: Conf. setinputformat (keyvaluetextinputformat. Class ); Jobconf. setoutputformat (myoutputformat. Class); setoutputformat: Textoutputformat: output to a plain text file in the format of key + "" + value. Nulloutputformat:/dev/null in hadoop, which sends the output to the black hole. Sequencefileoutputformat: Output sequencefile. The specific format is related to setoutputkeyclass and delimiter. In this case, the read format of sequencefileinputformat should be consistent with the output format of sequencefileoutputformat (Key + value format) encoding, multipletexto: output records to different files based on keys. You can rewrite dbinputformat and dboutputformat to read and output records from dB to DB. Example: Conf. setoutputformat (textoutputformat. Class );