Hadoop Full Distribution mode operation

Absrtact: This paper introduces the operation of Hadoop full distribution mode and realizes the cluster architecture in real sense.

Key words: Hadoop full distribution Mode file configuration

When using Hadoop to solve large data problems, we operate Hadoop in a fully distributed mode.

How to operate Hadoop based on full distribution mode, and build http://www.aliyun.com/zixun/aggregation/14119.html ">hadoop cluster?" The specific steps are as follows.

First step: Configure the Hosts file.

Execute command: vi/etc/hosts, modify all nodes, so that each other can be the host name resolution to IP, that is, to achieve the host name and IP binding.

Step two: Create a Hadoop running account

Creating an account running Hadoop on all nodes is not recommended because the root account is the most privileged and sometimes unexpected.

Step three: Configure SSH to achieve password-free login

Do the same for each node

First, check to see if SSH is installed

Execute commands sequentially: abound SSH

Abound sshd

Abound Ssh-keygen

Second, generate the SSH key pair

Execute command: ssh-keygen-r RSA

Finally, distribute the SSH public key

Copy each node's Authorized_keys contents to each other's file

Fourth step: Download and unzip the Hadoop installation package

Step Fifth: Configure Namenode, modify Hdfs-site.xml file

Sixth Step: Configure hadoop-env.sh

Description: The above three-step configuration can refer to pseudo distribution mode to manipulate Hadoop configuration

Step Seventh: Configure Masters and slaves files

Step eighth: Copy Hadoop to each node

Nineth Step: Format Namenode, start Hadoop

In view of the author's limited experimental environment and resources, Hadoop is currently operated based on pseudo distribution mode. Therefore, the above about the Hadoop full distribution mode of operation, the author has not yet tested, only through the relevant documents and data, a step aggregation. If there are problems or deficiencies, please correct me.

Original link: http://blog.csdn.net/wangloveall/article/details/20767161