Build a Hadoop cluster tips (3)

7 Yarn Installation Process

Install yarn on the basis of HDFS installation

1) Modify the Mapred-site.xml file to configure MapReduce

  <property>    <name>mapreduce.framework.name</name>    <value>yarn</value>  </property>  <property>    <name>mapreduce.jobhistory.address</name>    < value>master:10020</value>  </property>  <property>    <name> mapreduce.jobhistory.webapp.address</name>    <value>master:19888</value>  </ Property>

2) Modify yarn-env.sh

Modify Java_home value (export java_home=/usr/local/java/jdk1.7.0_79)

3) Modify Yarn-site.xml

Configure ResourceManager

  <property>    <name>yarn.resourcemanager.address</name>    <value>master:8032</ value>  </property>  <property>    <name>yarn.resourcemanager.scheduler.address </name>    <value>master:8030</value>  </property>  <property>    < Name>yarn.resourcemanager.resource-tracker.address</name>    <value>master:8031</value>  </property>  <property>    <name>yarn.resourcemanager.admin.address</name>    <value>master:8033</value>  </property>  <property>    <name> yarn.resourcemanager.webapp.address</name>    <value>master:8088</value>  </ Property>

Configure NodeManager

  <property>    <name>yarn.resourcemanager.hostname</name>    <value>master</value >    </property>  <property>    <name>yarn.nodemanager.aux-services</name>    <value>mapreduce_shuffle</value>  </property>  <property>    <name> Yarn.nodemanager.aux-services.mapreduce.shuffle. class</name>    <value>org.apache.hadoop.mapred.ShuffleHandler</value>  </property>

4) Start yarn

[Email protected]:/usr/local/hadoop# start-yarn.sh

Master has the following processes:

Slaves has the following processes:

5) Run WordCount verification

Hadoop jar Share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.0.jar Wordcount/input/output

Commands to run the jar package can be found on the official website, such as:

Run as follows:

View Results

At this point, the Hadoop cluster is fully configured.

Build a Hadoop cluster tips (3)