Hadoop installation Pseudo-distribution

Pseudo-Distributed Hadoop installation Summary

Prepare, in the configuration of the 9000 port for Hadoop, if there are other software using this port, it is recommended to replace the following configuration, to avoid errors. For example, PHP-FPM often uses port 9000.

First, download the JDK
Download Linux 8u73-64 bit version
Tar zxvf jdk-8u74-linux-x64.tar.gz-c/usr/local/

Second, download Hadoop
Tar zxvf hadoop-2.6.4.tar.gz-c/usr/local/

Configuration items:

vim/etc/profile.d/hadoop.sh
Export java_home=/usr/local/jdk1.8.0_74
Export JRE_HOME=/USR/LOCAL/JDK1.8.0_74/JRE
Export classpath=.: $JAVA _home/lib: $JRE _home/lib: $CLASSPATH
Export path= $JAVA _home/bin: $JRE _home/bin: $PATH
Export hadoop_home=/usr/local/hadoop-2.6.4
Export hadoop_prepath=/usr/local/hadoop-2.6.4
Export Hadoop_pid_dir=/usr/local/hadoop-2.6.4/pid
Export Yarn_pid_dir=/usr/local/hadoop-2.6.4/pid
Export Hadoop_mapred_pid_dir=/usr/local/hadoop-2.6.4/pid
Export path= $HADOOP _home/bin: $PATH

source/etc/profile.d/hadoop.sh

Vim Etc/hadoop/core-site.xml

<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop-2.6.4/tmp</value>
<description>abase for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>

Vim Etc/hadoop/yarn-site.xml

<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>
<property>
<description>the address of the Applications manager interface in the rm.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>the address of the scheduler interface.</description>
<name>yarn.resourcemanager.scheduler.address</name>
<value>${yarn.resourcemanager.hostname}:8030</value>
</property>
<property>
<description>the http address of the RM Web application.</description>
<name>yarn.resourcemanager.webapp.address</name>
<value>${yarn.resourcemanager.hostname}:8088</value>
</property>
<property>
<description>the https adddress of the RM Web application.</description>
<name>yarn.resourcemanager.webapp.https.address</name>
<value>${yarn.resourcemanager.hostname}:8090</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>${yarn.resourcemanager.hostname}:8031</value>
</property>
<property>
<description>the address of the RM admin interface.</description>
<name>yarn.resourcemanager.admin.address</name>
<value>${yarn.resourcemanager.hostname}:8033</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>

Vim Etc/hadoop/hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/usr/local/hadoop-2.6.4/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/usr/local/hadoop-2.6.4/dfs/data</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>

Vim Etc/hadoop/masters
localhost

Vim Etc/hadoop/slaves
localhost

mkdir tmp DFS dfs/name dfs/data

Bin/hdfs Namenode-format

sbin/start-dfs.sh
sbin/start-yarn.sh

Configure OK.

Hadoop installation Pseudo-distribution