Hadoop Common Configuration Meaning Memo

A number of configuration parameters are listed

Where the red configuration is required to configure parameters

Parameters	Take value	Notes
Fs.default.name	The URI of the Namenode.	HDFS://Host name/
Dfs.hosts/dfs.hosts.exclude	License/Reject Datanode list.	If necessary, use this file to control the licensed Datanode list.
Dfs.replication	Default: 3	Score for data replication
Dfs.name.dir	Example:/home/username/hadoop/namenode Default:/tmp	When this value is a comma-separated list of directories, the NameTable data is replicated to all directories for a redundant backup.
Dfs.data.dir	Example:/home/username/hadoop/datanode Default:/tmp	When this value is a comma-separated list of directories, the data is stored in all directories and is typically distributed across different devices.
Mapred.system.dir	The Map/reduce framework stores the HDFS path of the system files. Like/hadoop/mapred/system/.	This path is the path under the default file system (HDFS) and must be accessible from both the server and the client.
Mapred.local.dir	A comma-separated list of paths in the local file system where map/reduce temporary data is stored.	Multipath helps with disk I/O.
Mapred.tasktracker. {Map|reduce}.tasks.maximum	The maximum number of map/reduce tasks that can be run on a tasktracker, and these tasks will run separately.	The default is 2 (2 maps and 2 reduce), which can be changed based on hardware conditions.
Mapred.job.tracker	Jobtracker host (or IP) and port.	Host: Port.
Mapred.hosts/mapred.hosts.exclude	License/Reject Tasktracker list.	If necessary, use this file to control the licensed Tasktracker list.
Hadoop.job.history.user.location	Default: Mapred.output.dir/_logs/history can also be set to none to deactivate	History file record directory for job

Conf/slaves writes all slave machine names or IP

Namenode Remember that each file is mapped to a blockid. Each blockid corresponding block is copied to a different machine for an extra copy.

The default block for Hadoop is 64MB

To see if Namenode started: http://localhost:50070/

To see if Jobtracker started: http://localhost:50030/