[Spark Asia Pacific Research Institute Series] the path to spark practice-Chapter 1 building a spark cluster (step 5) (3)

Step 4 modify the configuration file slaves as follows:

We set the slave nodes in the hadoop cluster to sparkworker1 and sparkworker2, and modify the slaves file content:

Step 5 modify the profile core-site.xml, as shown below:

Modify the content of the core-site.xml file:

The above is the minimal configuration of the core-site.xml file, the core-site.xml configuration can refer:

Http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml

Step 6 modify the profile hdfs-site.xml, as shown below:

Modify the content of the hdfs-site.xml file:

The above is the minimum configuration of the hdfs-site.xml file, the hdfs-site.xml configuration course reference:

Http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml

Step 7 modify the profile mapred-site.xml, as shown below:

Copy A mapred-site.xml.template named as a mapred-site.xml, open the mapred-site.xml

Modify the content of the mapred-site.xml file:

The above is mapred-site.xml minimal configuration, mapred-site.xml configuration can refer:

Http://hadoop.apache.org/docs/r2.2.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

Step 7 modify the profile yarn-site.xml, as shown below:

Modify the content of the yarn-site.xml:

The above content is the minimal configuration of the yarn-site.xml, the content of the yarn-site.xml file configuration can be referred:

Http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

[Spark Asia Pacific Research Institute Series] the path to spark practice-Chapter 1 building a spark cluster (step 5) (3)