Hadoop from Getting started to mastering (i): Preparing for Hadoop environment setup

Hello everyone, I am Stefan, starting today to bring you a detailed Hadoop learning tutorial, you can follow my tutorial step by step into the development of cloud computing, OK, nonsense, we started the first: Hadoop environment.

The beginning of everything is difficult, this is not a blow. Many people in the initial environment to build up the problem, and everyone's platform and there are differences, it is more caused by the probability of the occurrence of problems, here I try to relate to a variety of situations, I use the Mac OS 10.9.2, the process of building and using Linux platforms such as Ubuntu and CentOS is the same process.

(a) First, we need to go to the official Internet Download Hadoop installation package, address, here I choose is hadoop-2.2.0.tar.gz, into the Hadoop2.0 era, there is no reason not to keep up with the trend of the times. I downloaded to the local download directory, decompression tar-zxf hadoop-2.2.0.tar.gz.

(b) We need to install the Java environment, because I am a mac system, the download eclipse process has been automatically installed in the Java environment, if you do not know whether to install the Java environment, you can enter the terminal: Java-version view, as seen below:

Java version "1.6.0_65"
Java (tm) SE Runtime Environment (build 1.6.0_65-b14-462-11m4609)
Java HotSpot (tm) 64- Bit Server VM (build 20.65-b04-462, Mixed mode)

It proves that the Java environment has been installed, if the Java environment is not installed child paper can download the JDK for installation, this step is relatively simple, here is not redundant. Also set the JDK as the system default JDK and set the java_home environment.

(iii) After the two steps have been completed, the next step is to set up SSH password-free login. Because the Hadoop cluster is hundreds of thousands of scales, if each machine requires a password login, then the workload is quite large, so we need to set up SSH authentication key login, so do not need to manually enter the password. First we install Ssh,ubuntu can use such as: sudo apt-get install ssh installed, my Mac system has been installed by default SSH, so do not need to install, then set up password-free login, terminal input the following command:

Ssh-keygen-t Dsa-p "-F ~/.SSH/ID_DSA

In this case, the ID_DSA and id_dsa.pub two private keys and public keys are generated under the User/yourname/.ssh folder, and next we need to add the public key to the authorization key file (Authorized_key) and enter the following command:

Cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

The purpose of this command is to write the information inside the id_dsa.pub to the last side of the Authorized_key, now that the password-free login has been configured to complete.

Before the test, the Mac system users also need to note that SSH remote login By default is disable, we need to open it first. In the settings

Find sharing options

Then, in the Share panel, select Telnet

Then we modify the permissions of the. SSH folder: Knock in the terminal

Chmod-r ~/.ssh/

Now we can use SSH login, we test, in the terminal to knock

Ssh-version

Show

OPENSSH_6.2P2, Osslshim 0.9.8r 8 Dec bad
escape character ' rsion '.

See this and you're done. Next we try remote login, we log in to our host to try

SSH localhost

Show

The authenticity of host ' 127.0.0.1 (127.0.0.1) ' can ' t be established.
RSA key fingerprint is 0b:41:26:f6:9e:d6:b4:79:e9:26:0c:6d:7f:b4:b4:81.
Is you sure want to continue connecting (yes/no)? Yes
warning:permanently added ' 127.0.0.1 ' (RSA) to the list of known hosts.
Last Login:sat Mar-14:56:15 from localhost

The first login is required to enter Yes to determine the key is correct, the last word to prove that we have logged in successfully ~

Today's Hadoop learning is here, thank you for your attention, I will continue to update my blog, to bring you more learning about Hadoop tutorial.

Reprint Please specify source: http://blog.csdn.net/guoshenglong11/article/details/22430755