Hadoop Common Configuration Meaning Memo

Source: Internet
Author: User

A number of configuration parameters are listed

Where red is configured to have parameters configured

Parameter value Comment

Fs.default.name The URI of the NameNode. HDFS://Host name/
Dfs.hosts/dfs.hosts.exclude License/Deny Datanode list. If necessary, use this file to control the list of licensed Datanode.
Dfs.replication Default: 3 Score for data replication
Dfs.name.dir

Example:/home/username/hadoop/namenode

Default:/tmp

When this value is a comma-delimited list of directories, the NameTable data will be copied to all directories for redundant backups.
Dfs.data.dir

Example:/home/username/hadoop/datanode

Default:/tmp

When this value is a comma-separated list of directories, the data will be stored in all directories, usually distributed on different devices.
Mapred.system.dir Map/reduce the HDFS path to the framework storage system files. Like/hadoop/mapred/system/. This path is the path under the default file system (HDFS) and must be accessible from both the server and the client.
Mapred.local.dir A comma-delimited list of paths under the local file system, where temporary data is stored map/reduce. Multipath helps take advantage of disk I/O.
Mapred.tasktracker. {Map|reduce}.tasks.maximum The maximum number of map/reduce tasks that can be run on a tasktracker, and these tasks will run concurrently. The default is 2 (2 map and 2 reduce), which can be changed according to the hardware condition.
Mapred.job.tracker The host (or IP) and port of the Jobtracker. Host: Port.
Mapred.hosts/mapred.hosts.exclude License/Deny Tasktracker list. If necessary, use this file to control the list of licensed Tasktracker.
Hadoop.job.history.user.location

Default: Mapred.output.dir/_logs/history

can also be set to none to deactivate

History file record directory for the job

Conf/slaves write the machine name or IP of all slave

Namenode Remember that each file is mapped into a blockid. Each blockid corresponding block is copied to a different machine.

The default block for Hadoop is 64MB

See if Namenode starts: http://localhost:50070/

See if Jobtracker starts: http://localhost:50030/


Reprint: http://eryk.iteye.com/blog/792962

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.