Hadoop single node pseudo-distributed construction

Hadoop version: 2.7.1 jdk:1.7 os:ubuntu 14.04

1. Open SSH password-free login

Ssh-keygen->> Authorized_keys

Here do not repeat, do not open please Google, Baidu (build Java environment also)

2, modify Core-site.xml (path:etc/hadoop/)

<Configuration>     < Property>          <name>Fs.defaultfs</name>          <value>hdfs://localhost:9000</value>      </ Property></Configuration>

Meaning: Receives the RPC port of the client connection for obtaining file system metadata information.

3, modify the Hdfs-site.xml

<Configuration>        < Property>            <name>Dfs.replication</name>            <value>1</value>        </ Property></Configuration>

Meaning: Backup has only one copy

4. Set environment variables and refresh profile

Export java_home=/usr/lib/javaexport jre_home=${java_hmome}/jreexport CLASSPATH=.:  ${java_hmoe}/libexport hadoop_home=/usr/lib/hadoopexport PATH=${hadoop_home}/bin :${hadoop_home}/sbin:${java_home}/bin:${path}

5. Format HDFs

HDFs Namenode-format

6. Start the cluster

start-dfs.sh

7. Verification

You can use the JPS command to verify that there are processes

　　

can use URL access, port is 50070, for example: 10.0.6.206:50070

　　

Configure yarn

1, modify the Mapred-site.xml

CP Mapred-site.xml.template Mapred-site.xml

To add a configuration:

<  Property >        < name >mapreduce.framework.name</name>        <value  >yarn</value>    </Property  >

2, modify the Yarn-site.xml

  <  Property >        < name >yarn.nodemanager.aux-services</name>        <  Value>mapreduce_shuffle</value>    </  Property>

3. Start yarn

Start-yarn.xml

4. Verification

Verify by URL:

Later I will try to use Docker to build a pseudo-distribution and cluster, to be continued ...

Hadoop single node pseudo-distributed construction