Construction of Hadoop environment with pseudo-distributed mode

First, this article explains:
This test is a pseudo-distributed build on a virtual machine system. Hadoop pseudo-distributed mode is the simulation of Hadoop distributed on a single machine, distributed on a single machine is not really pseudo-distributed, but the use of threading simulation distributed. Hadoop itself is indistinguishable from pseudo-distributed and distributed, and the two configurations are similar. The only difference is that pseudo-distribution is configured on a single machine, and the data node and the name node are all machines.

Second, the environment description:
Operating system: Cent OS 6.7
Hadoop version: hadoop-2.7.2
JDK version: jdk1.8_77
Note: Built on an independent model, refer to the article: the standalone model of the Hadoop environment.

Three, the basic Environment configuration
1. Set up different configuration folders for different modes under ${hadoop_home}/etc/
Standalone mode: Hadoop_alone (recommended)
Pseudo distribution mode: Hadoop_presudo (recommended)
Fully distributed mode: Hadoop_cluster (recommended)
2, the parameter configuration directory to specify the way
A, default mode: ${hadoop_home}/etc/hadoop/*.xml
B. Specify the configuration directory through the startup parameters
Start-dfs.sh–config=/path
C, through the environment variable Hadoop_conf_dir=/path (/etc/profile)
Note: Because of the pseudo distribution pattern: The ${hadoop_home}/etc/hadoop_presudo i configured here
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413040129321.png "alt=" 20160413040129321.png "/>
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413040147148.png "alt=" 20160413040147148.png "/>

Third, the environment construction
1, Configuration ${hadoop_home}/etc/hadoop_presudo/core-site.xml
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413040608831.png "alt=" 20160413040608831.png "/>

2, Configuration ${hadoop_home}/etc/hadoop_presudo/hdfs-site.xml
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413040716768.png "alt=" 20160413040716768.png "/>


3, Configuration ${hadoop_home}/etc/hadoop_presudo/mapred-site.xml
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413040805330.png "alt=" 20160413040805330.png "/>


4, Configuration ${hadoop_home}/etc/hadoop_presudo/yarn-site.xml
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413040859775.png "alt=" 20160413040859775.png "/>


5. Format HDFs File system
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413040923742.png "alt=" 20160413040923742.png "Width="/>

650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413041032792.png "alt=" 20160413041032792.png "Width="/>

6. Start the daemon for HDFs and Yran (start-dfs.sh,start-yarn.sh)
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413041057741.png "alt=" 20160413041057741.png "Width="/>

650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413041134146.png "alt=" 20160413041134146.png "Width="/>
Note: The stop process is followed by yarn close and then close HDFs, do not format the name node frequently


7. Operation of files in HDFs via FS
A. Create directory: Hadoop fs-mkdir-p/user/ubuntu
B. Display directory: Hadoop fs-ls-r/
......



8. Visit: http://IP:50070
650) this.width=650; "src=" Http://www.itroad.org/uploadfile/2016/0413/20160413041224922.png "alt=" 20160413041224922.png "Width="/>

Construction of Hadoop environment with pseudo-distributed mode