Hadoop叢集環境安裝部署

標籤：hadoop webseven 安裝部署

1.環境準備:

安裝Centos6.5的作業系統

下載hadoop2.7檔案

下載jdk1.8檔案

2.修改/etc/hosts檔案及配置互信:

在/etc/hosts檔案中增加如下內容:

192.168.1.61 host61

192.168.1.62 host62

192.168.1.63 host63

配置好各伺服器之間的ssh互信

3.添加使用者,解壓檔案並配置環境變數:

useradd hadoop

passwd hadoop

tar -zxvf hadoop-2.7.1.tar.gz

mv hadoop-2.7.1 /usr/local

ln -s hadoop-2.7.1 hadoop

chown -R hadoop:hadoop hadoop-2.7.1

tar -zxvf jdk-8u60-linux-x64.tar.gz

mv jdk1.8.0_60 /usr/local

ln -s jdk1.8.0_60 jdk

chown -R root:root jdk1.8.0_60

echo ‘export JAVA_HOME=/usr/local/jdk‘ >>/etc/profile

echo ‘export PATH=/usr/local/jdk/bin:$PATH‘ >/etc/profile.d/java.sh

4.修改hadoop設定檔:

1)修改hadoop-env.sh檔案:

cd /usr/local/hadoop/etc/hadoop/hadoop-env.sh

sed -i ‘s%#export JAVA_HOME=${JAVA_HOME}%export JAVA_HOME=/usr/local/jdk%g‘ hadoop-env.sh

2)修改core-site.xml,在最後添加如下內容:

<configuration>  

<property>

<name>fs.default.name</name>

<value>hdfs://host61:9000/</value>

</property>

<property>

<name>hadoop.tmp.dir</name>

<value>/home/hadoop/temp</value>

</property>

</configuration>  

3)修改hdfs-site.xml檔案:

<configuration>    

<property>    

<name>dfs.replication</name>    

<value>3</value>    

</property>    

</configuration>

4)修改mapred-site.xml

<configuration>    

<property>    

<name>mapred.job.tracker</name>    

<value>host61:9001</value>    

</property>    

</configuration>

5)配置masters

host61

6)配置slaves

host62

host63

5.用同樣的方式配置host62及host63

6.格式化Distributed File System

/usr/local/hadoop/bin/hadoop -namenode format

7.運行hadoop

1)/usr/local/hadoop/sbin/start-dfs.sh

2)/usr/local/hadoop/sbin/start-yarn.sh

8.檢查:

[[email protected] sbin]# jps

4532 ResourceManager

4197 NameNode

4793 Jps

4364 SecondaryNameNode

[[email protected] ~]# jps

32052 DataNode

32133 NodeManager

32265 Jps

[[email protected] local]# jps

6802 NodeManager

6963 Jps

6717 DataNode

本文出自 “webseven” 部落格，請務必保留此出處http://webseven.blog.51cto.com/4388012/1699359

Hadoop叢集環境安裝部署