ubuntu16.04+hadoop2.7.4 Environment to build _hadoop

Reprint please indicate source origin: http://www.cnblogs.com/lighten/p/6106891.html
Installation of 1.JDK

1. Go to Oracle website to download, link contact this. You can also use the wget http://download.oracle.com/otn-pub/java/jdk/8u111-b14/jdk-8u111-linux-x64.tar.gz to install the folder location where you are performing this command.

2. Decompression TAR-ZXVF jdk-8u111-linux-x64.tar.gz

3. Move to the position you want:

Mkdir/usr/lib/jdk

MV jdk1.8.0_111/usr/lib/jdk/jdk1.8

4. Set Environment variables:

Scenario One: Modify the global configuration file for all users: Vim/etc/profile

Export java_home=/usr/lib/jdk/jdk1.8

Export JRE_HOME=${JAVA_HOME}/JRE

Export Classpath=.:${java_home}/lib:${jre_home}/lib

Export Path=.:${java_home}/bin: $PATH

Scenario Two: Modify the current user profile to function only for the current user: Vim ~/.BASHRC settings are the same

5. Make the modified configuration immediately effective: source/etc/profile or source ~/.BASHRC

6. Check to see if the installation was successful: Java-version

2. Configure SSH and password-free login

With:
Hadoop needs to use SSH to log in, and Linux needs to install SSH. The client is already installed, just install the server:

sudo apt-get install openssh-server test login native ssh localhost input yes should be able to log in. But each input is more cumbersome, if it is a cluster that is a disaster, so to configure the way to password-free landing.

A total of three steps:

1. Generate public key private key ssh-keygen-t RSA, files will be generated under the ~/.ssh folder Id_rsa: Private key, id_rsa.pub: Public key

Ssh-keygen-t Dsa-p '-F ~/.SSH/ID_DSA

2. Import public key to authentication file, change permissions:

1) Import this machine:

Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

2) Import Server:

First copy the public key to the server:

#scp ~/.ssh/id_rsa.pub xxx@host:/home/xxx/id_rsa.pub

Id_rsa.pub 100% 389 0.4kb/s 00:00

The public key is then imported into the authentication file, which is done on the server:

Cat ~/id_rsa.pub >> ~/.ssh/authorized_keys

Finally, change permissions on the server:

chmod ~/.ssh
chmod ~/.ssh/authorized_keys

3) Test: SSH localhost the first time you need to enter Yes, then you do not need.

Installation of 3.Hadoop

1. Download the Hadoop installation package, download the address: click here. Download binary on the line. You can also use the wget command to download.

We suggest the following mirrorsite for your download:

Http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.4/hadoop-2.7.4.tar.gz

Other mirror sites are suggested below. Please use the backup mirrors Onlyto download PGP and MD5 signatures toverify your downloads or if no othermirrors are King.

#wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.4/hadoop-2.7.4.tar.gz

2. Unpack and move to the folder you want to place

TAR-ZVXF hadoop-2.7.4.tar.gz

mv./hadoop-2.7.4.tar.gz/opt/hadoop

3. Create Hadoop users and groups and grant execution permissions

sudo addgroup Hadoop

sudo usermod-a-g Hadoop xxx #将当前用户加入到hadoop组

sudo visudo #将hadoop组加入到sudoer (from: Add Sudoer)

After Root all= (all), the Hadoop all= (all) all sudo chmod-r 755/etc/opt/hadoop

sudo chown-r xxx:hadoop/etc/opt/hadoop//Otherwise SSH will deny access

These are the general needs of the operation, this article also made other configuration, if you encounter problems can see, is not due to these configuration caused by: click here.

4. Modify the configuration file, as with the JDK installation, you can choose which file to modify. Modify/etc/profile here

Export hadoop_home=/etc/opt/hadoop2.7.4 (note.) According to your own path)

Export Path=.:${java_home}/bin:${hadoop_home}/bin: $PATH

Source/etc/profile