Install hadoop and centoshadoop in Centos 7.0

Install hadoop and centoshadoop in Centos 7.0

I. installation environment
Hardware: Virtual Machine
Operating System: Centos 7.0 64-bit
IP: 192.168.120.150

Hadoop-2.7.0.tar.gz link: http://pan.baidu.com/s/1eRT0tk2 password: ymim

Jdk-8u45-linux-x64.tar.gz link: http://pan.baidu.com/s/1eSaRUGa password: f4ue

Ii. Install JDK
1. Download jdk-8u45-linux-x64.tar.gz and decompress it to/usr/java/jdk1.8.0 _ 45.
2. Add the following configuration in/root/. bash_profile:

Export JAVA_HOME =/usr/java/jdk1.8.0 _ 45
Export PATH = $ JAVA_HOME/bin: $ PATH

3. Make the environment variable take effect. # source ~ /. Bash_profile
4. Installation verification # java-version
Java version "1.8.0 _ 45"
Java (TM) SE Runtime Environment (build 1.8.0 _ 45-b14)
Java HotSpot (TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

3. Configure SSH login without a password

$ Ssh-keygen-t dsa-p'-f ~ /. Ssh/id_dsa

$ Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys

Verify ssh, # ssh localhost
You do not need to enter a password to log on.

If you are in the form of a cluster, you can refer to the http://blog.csdn.net/se7en_q/article/details/47671425

4. Install Hadoop2.7.0
1. Download Hadoop2.7.0
2. Unzip and install
1), copy the hadoop-2.7.0.tar.gz to the/usr/hadoop directory,
Then # tar-xzvf hadoop-2.7.0.tar.gz unzip, unzip the Directory:/usr/hadoop/hadoop-2.7.0
2) In the/usr/hadoop/directory, create the tmp, hdfs/name, hdfs/data directory and execute the following command
# Mkdir/usr/hadoop/tmp
# Mkdir/usr/hadoop/hdfs
# Mkdir/usr/hadoop/hdfs/data
# Mkdir/usr/hadoop/hdfs/name

3), set the environment variable, # vi ~ /. Bash_profile

# Set hadoop path
Export hadooop_home =/usr/hadoop/hadoop-2.7.0
Export PATH = $ PATH: $ HADOOP_HOME/bin

4) make the environment variable take effect, $ source ~ /. Bash_profile

3. Hadoop Configuration
Go to the $ HADOOP_HOME/etc/hadoop directory and configure the hadoop-env.sh. The configuration files involved are as follows:
Hadoop-2.7.0/etc/hadoop/hadoop-env.sh.
Hadoop-2.7.0/etc/hadoop/yarn-env.sh.
Hadoop-2.7.0/etc/hadoop/core-site.xml.
Hadoop-2.7.0/etc/hadoop/hdfs-site.xml.
Hadoop-2.7.0/etc/hadoop/mapred-site.xml.
Hadoop-2.7.0/etc/hadoop/yarn-site.xml.

1) Configure hadoop-env.sh

# The java implementation to use.
# Export JAVA_HOME =$ {JAVA_HOME}
Export JAVA_HOME =/usr/java/jdk1.8.0 _ 45

2) Configure yarn-env.sh

# Export JAVA_HOME =/home/y/libexec/jdk1.7.0/
Export JAVA_HOME =/usr/java/jdk1.8.0 _ 45

3) Configure core-site.xml
Add the following Configuration:

<Configuration>
<Property>
<Name> fs. default. name </name>
<Value> hdfs :/// localhost: 9000 </value>
<Description> hdfs uri, file system: // namenode ID: port number </description>
</Property>

<Property>
<Name> hadoop. tmp. dir </name>
<Value>/usr/hadoop/tmp </value>
<Description> Local hadoop Temporary Folder on namenode </description>
</Property>
</Configuration>

4), configure hdfs-site.xml
Add the following configuration

<Configuration>
<! -Hdfs-site.xml -->
<Property>
<Name> dfs. name. dir </name>
<Value>/usr/hadoop/hdfs/name </value>
<Description> store hdfs namespace metadata on namenode </description>
</Property>

<Property>
<Name> dfs. data. dir </name>
<Value>/usr/hadoop/hdfs/data </value>
<Description> physical storage location of data blocks on datanode </description>
</Property>

<Property>
<Name> dfs. replication </name>
<Value> 1 </value>
<Description> Number of replicas. The default value is 3, which must be smaller than the number of datanode machines. </description>
</Property>
</Configuration>

5), configure mapred-site.xml
Add the following Configuration:

<Configuration>
<Property>
<Name> mapreduce. framework. name </name>
<Value> yarn </value>
</Property>
</Configuration>

6), configure yarn-site.xml
Add the following Configuration:

<Configuration>
<Property>
<Name> yarn. nodemanager. aux-services </name>
<Value> mapreduce_shuffle </value>
</Property>
<Property>
<Name> yarn. resourcemanager. webapp. address </name>
<Value> 192.168.120.150: 8099 </value>
</Property>
</Configuration>

4. Hadoop startup
1) format namenode

Hadoop/hadoop-2.7.0/sbin/cd/

./Hdfs namenode-format

2) Start the NameNode and DataNode daemon

Start-dfs.sh

3) Start the ResourceManager and NodeManager daemprocesses.

Sbin/start-yarn.sh

5. Start verification.
1) execute the jps command. The following process indicates that Hadoop is started normally.
# Jps
71072 ResourceManager
71171 NodeManager
Jps 72690
6. log on to hadoop
Http: 192.168.120.150: 8099