標籤:hadoop
1.安裝前,準備三台CENTOS 6.6系統的主機或虛機,並且關閉防火牆及selinux.
2.按如下表格配置IP地址,修改hosts檔案及本機名
192.168.199.21 hadoop21 Master
102.168.199.22 hadoop22 Slave1
192.168.199.23 hadoop23 Slave2
650) this.width=650;" src="http://s14.sinaimg.cn/mw690/002nJwOegy6UlUhMq6xcd&690" width="632" height="359" name="image_operate_78961438613426977" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
同理修改Slave1,Slave2的IP地址,hosts檔案及本機名。
3.安裝ORACLE JDK
先卸載本機openJDK,使用rpm -qa|grep java查看,然後用rpm -e 卸載
從oracle網站找到最新JDK,我這選擇了JDK8
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
下載好以後解壓,並移至/usr/java,如果沒有可以mkdir 建立。
tar -xzvf jdk-8u51-linux-x64.gz
mv jdk1.8.0_51 /usr/java
vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_51
export CLASSPATH=.:$JAVA_HOME/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
source /etc/profile
4.在各節點分別建立Hadoop運行帳號grid,並設定密碼
5.配置SSH免密碼登陸。
在各節點分別以grid使用者名稱產生兩個密鑰檔案,一個是私密金鑰id_rsa,另一個是公開金鑰id_rsa.pub
ssh-keygen -t rsa -f ~/.ssh/id_rsa
然後在hadoop21上
cp /home/grid/.ssh/id_rsa.pub /home/grid/.ssh/authorized_keys
scp hadoop22:/home/grid/.ssh/id_rsa.pub pubkeys22
scp hadoop23:/home/grid/.ssh/id_rsa.pub pubkeys23
cat pubkeys22 >> /home/grid/.ssh/authorized_keys
cat pubkeys23 >> /home/grid/.ssh/authorized_keys
rm pubkeys22
rm pubkeys23
最後分發authorized_keys 到各節點
scp /home/grid/.ssh/authorized_keys hadoop22:/home/grid/.ssh
scp /home/grid/.ssh/authorized_keys hadoop23:/home/grid/.ssh
6.在Master機下載並解壓Hadoop2.7.1(使用grid使用者名稱)
找到最近的hadoop鏡像,使用wget下載2.7.1
wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz
解壓hadoop-2.7.1.tar.gz
tar -xzvf hadoop-2.7.1.tar.gz
cd hadoop-2.7.1
建立tmp,dfs,dfs/data,dfs/name
7.修改設定檔
修改hadoop-env.sh
export JAVA_HOME=/usr/java/jdk1.8.0_51
650) this.width=650;" src="http://s3.sinaimg.cn/mw690/002nJwOegy6UlVcJVkK62&690" width="464" height="322" name="image_operate_41561438614255823" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
650) this.width=650;" src="http://s8.sinaimg.cn/mw690/002nJwOegy6UlVhJ5xJ07&690" width="567" height="498" name="image_operate_85511438614256230" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
650) this.width=650;" src="http://s11.sinaimg.cn/mw690/002nJwOegy6UlVcQwkOca&690" width="581" height="327" name="image_operate_8841438614256447" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
650) this.width=650;" src="http://s5.sinaimg.cn/mw690/002nJwOegy6UlVcV90844&690" name="image_operate_63271438614256703" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
650) this.width=650;" src="http://s13.sinaimg.cn/mw690/002nJwOegy6UlVcXuiM0c&690" width="612" height="300" name="image_operate_7921438614470465" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
8. 分發至各Salve節點
scp -r /home/grid/hadoop-2.7.1 hadoop22:/home/grid
scp -r /home/grid/hadoop-2.7.1 hadoop23:/home/grid
9.Master機格式化namenode
cd /home/grid/hadoop-2.7.1
./bin/hdfs namenode -format
10.啟動Hadoop
650) this.width=650;" src="http://s14.sinaimg.cn/mw690/002nJwOegy6UlVBlJbD4d&690" width="675" height="424" name="image_operate_42161438614552203" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
650) this.width=650;" src="http://s2.sinaimg.cn/mw690/002nJwOegy6UlVESzBLa1&690" width="675" height="424" name="image_operate_83381438614610802" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
11.驗證是否成功
Master機應該啟動NameNode,SecondaryNameNode,ResourceManager
Slave機應該啟動DataNode,NodeManager
650) this.width=650;" src="http://s4.sinaimg.cn/mw690/002nJwOegy6UlVIT0qf83&690" width="670" height="516" name="image_operate_33011438614651489" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
650) this.width=650;" src="http://s13.sinaimg.cn/mw690/002nJwOegy6UlVIXVHm6c&690" width="690" height="374" name="image_operate_32881438614709117" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
650) this.width=650;" src="http://s5.sinaimg.cn/mw690/002nJwOegy6UlVJ1NcMc4&690" width="690" height="233" name="image_operate_87201438614707089" alt="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" title="centos <wbr>6.6 <wbr>hadoop <wbr>2.7.1 <wbr>完全分布式安裝部署" style="margin:0px;padding:0px;border:0px;list-style:none;" />
本文出自 “夢想照進現實” 部落格,請務必保留此出處http://lookingdream.blog.51cto.com/5177800/1835082
centos 6.6 hadoop 2.7.1 完全分布式安裝部署