Hadoop偽分布式搭建CentOS

標籤：

所需軟體及版本：jdk-7u80-linux-x64.tar.gzhadoop-2.6.0.tar.gz1.安裝JDK    Hadoop 在需在JDK下運行,注意JDK最好使用Oracle的否則可能出bug2.建立使用者    [[email protected] ~]# groupadd hadoop    [[email protected] ~]# useradd -g hadoop hadoop    [[email protected] ~]# passwd hadoop 3.配置SSH    確認SSH的用戶端和服務端已安裝    [[email protected] ~]$ rpm -qa|grep ssh    libssh2-1.4.2-1.el6.x86_64    openssh-clients-5.3p1-94.el6.x86_64    openssh-server-5.3p1-94.el6.x86_64    openssh-5.3p1-94.el6.x86_64    [[email protected] ~]$     配置SSH免密碼登陸    [[email protected] ~]$ mkdir ~/.ssh    [[email protected] ~]$ cd ~/.ssh/                            # 若沒有該目錄，請先執行一次ssh localhost      [[email protected] .ssh]$ ssh-keygen -t dsa                  # 會有提示，都按斷行符號就可以      [[email protected] .ssh]$ cat id_dsa.pub >> authorized_keys  # 加入授權      [[email protected] .ssh]$ chmod 600 ./authorized_keys        # 修改檔案許可權，如果不改，無法通過，原因好像是cent os的許可權驗證比較嚴格    [[email protected] .ssh]$ssh MyDB01                          # 測試4.安裝hadoop    下載    [[email protected] ~]$ wget  http://archive.apache.org/dist/hadoop/core/hadoop-2.6.0/hadoop-2.6.0.tar.gz        解壓    [[email protected] hadoop]#tar -zxvf hadoop-2.6.0.tar.gz -C /usr/local    修改許可權    [[email protected] local]# mv hadoop-2.6.0/ hadoop/          #變更檔夾名稱      [[email protected] local]# chown -R hadoop:hadoop ./hadoop   #修改許可權    修改環境變數    [[email protected] ~]$ vi .bash_profile    export HADOOP_HOME=/usr/local/hadoop      export HADOOP_INSTALL=$HADOOP_HOME      export HADOOP_MAPRED_HOME=$HADOOP_HOME      export HADOOP_COMMON_HOME=$HADOOP_HOME      export HADOOP_HDFS_HOME=$HADOOP_HOME      export YARN_HOME=$HADOOP_HOME      export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native      export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin      export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64     export PATH=$JAVA_HOME/bin:$PATH      export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar  5.配置偽分布式    #手動建立根目錄/hadoop  屬主:hadoop    1、core-site.xml檔案       <configuration>        <property>              <name>hadoop.tmp.dir</name>              <value>file:/hadoop/tmp</value>          </property>          <property>              <name>fs.defaultFS</name>              <value>hdfs://MyDB01:9000</value>          </property>        </configuration>       2、hdfs-site.xml檔案       <configuration>        <property>              <name>dfs.replication</name>              <value>1</value>          </property>          <property>              <name>dfs.namenode.name.dir</name>              <value>file:/hadoop/tmp/dfs/name</value>          </property>          <property>              <name>dfs.datanode.data.dir</name>              <value>file:/hadoop/tmp/dfs/data</value>          </property>         </configuration> 6.對檔案系統HDFS格式化操作    hadoop namenode -format

 

Hadoop偽分布式搭建CentOS