learn hadoop from scratch

The advantage of the MapReduce framework is the ability to run mapper and reducer tasks in parallel in the cluster, how to determine the number of mapper and reducer, or how to programmatically control the number of mapper and reducer that the job starts? In the mapper and reducer of the Hadoop-2.4.1 study, it was mentioned that the number of recommended reducer is (0.95~1.75) * Number of nodes * Maximum number of containers per node, and method Job.s

HDFs, as a distributed file system, is the foundation of all these projects. The analysis of HDFs is good for understanding other systems. Since Hadoop's HDFs and MapReduce are the same project, we put them together for analysis.If you take the whole Hadoop as a class in Java, then HDFs is the static variable of this class, and the other projects are the methods in Hadoop.HDFsImplementation of Hdfs,hadoop D

Writable that writes an instance with it's class name.* Handles arrays, strings and primitive types without a writable wrapper.*/Public class Objectwritable implements writable, configurable { Private Class Declaredclass;Private Object instance;Private Configuration conf; --------------------------------------------------------------------------------------------------------------- ---------------------------- Writable writable = Writablefactories.newinstance (Instanceclass, conf);Writable.rea

;Private Final Call Saslcall = new call (Sasl_callid, NULL, this);Private final Bytearrayoutputstream saslresponse = new Bytearrayoutputstream ();Private Boolean usewrap = false; 6, Exceptionshandler manages Exception groups for special handling e.g., terse Exception Group for concise logging messages 7, Handles queued calls. 8, listens on the socket. Creates jobs for the handler threads Inside the listener there is an inner class, Reader The corresponding properties are Private Serversocketcha