Linux Configuration for Hadoop

I. New Hadoop group and user (password: hadoop)

[

Email protected] home]# groupadd Hadoop[ [email protected] home]# useradd-g HadoopHadoop [[ Email protected] home]# passwd hadoop changing password for user Hadoop. New Password: Bad Password:it are based on a dictionary word Bad Password:is too simple Retype new Password: Passwd:all authentication Tokens updated successfully. [email protected] home]#

Second, timing service (time synchronization)

[Email protected] home]# crontab–e

[Email protected] home]# crontab-l

0 1 * * */usr/sbin/ntpdate cn.pool.ntp.org

Third, Jdk-6u45-linux-i586.bin installation (no X permissions, modify permissions after execution)

[Email protected] java]# pwd

/usr/local/java

[email protected] java]# LL

Total 130600

-rwxrw-rw-. 1 root root 61927560 June 7 hadoop-1.1.2.tar.gz

-rw-r--r--. 1 root root 71799552 Oct 14:33 jdk-6u45-linux-i586.bin

[Email protected] java]# chmod u+x jdk-6u45-linux-i586.bin

[email protected] java]# LL

Total 130600

-rwxrw-rw-. 1 root root 61927560 June 7 hadoop-1.1.2.tar.gz

-rwxr--r--. 1 root root 71799552 Oct 14:33 jdk-6u45-linux-i586.bin

[Email protected] java]#./jdk-6u45-linux-i586.bin

Configure environment variables (do not configure in profile, create a new java.sh file, configure the Java environment variables, the profile file will automatically load the java.sh file)

[Email protected] jdk1.6.0_45]# pwd

/usr/local/java/jdk1.6.0_45

[Email protected] jdk1.6.0_45]# vi/etc/profile.d/java.sh

[Email protected] jdk1.6.0_45]#

[[Email protected] jdk1.6.0_45]# java

Bash:java:command not found

[[email protected] jdk1.6.0_45]# source/etc/profile (make java.sh file configuration effective)

[Email protected] jdk1.6.0_45]# java-version

Java Version "1.6.0_45"

Java (TM) SE Runtime Environment (build 1.6.0_45-b06)

Java HotSpot (TM) Client VMs (build 20.45-b01, mixed mode, sharing)

[Email protected] jdk1.6.0_45]# javac-version

Javac 1.6.0_45

[Email protected] jdk1.6.0_45]#

Iv. Host Name Modification

1, vi/etc/sysconfig/network
Change hostname to Node
Verify: Restart Linux
Or: the hostname hostname (which is set to hostname temporarily, fails after a reboot) is best performed to
Next Source/etc/profile

2, [[email protected] jdk1.6.0_45]# vi/etc/sysconfig/network

[Email protected] jdk1.6.0_45]# hostname
Localhost.localdomain
[Email protected] jdk1.6.0_45]#
You need logout here once, the hostname will take effect

Five, the Hosts file modification (can ping pass on success)

[Email protected] desktop]# vi/etc/hosts

[[email protected] desktop]# Ping Master
PING Master (192.168.209.100) bytes of data.
Bytes from master (192.168.209.100): icmp_seq=1 ttl=64 time=0.488 ms
Bytes from master (192.168.209.100): icmp_seq=2 ttl=64 time=0.083 ms
^c
---Master ping Statistics---
2 packets transmitted, 2 received, 0% packet loss, time 1374ms
RTT Min/avg/max/mdev = 0.083/0.285/0.488/0.203 ms
[Email protected] desktop]#

Vi. Firewall shutdown

[[Email protected] desktop]# service iptables status
Table:filter
Chain INPUT (Policy ACCEPT)
Num Target prot opt source destination
1 ACCEPT All – 0.0.0.0/0 0.0.0.0/0 State related,established
2 ACCEPT ICMP--0.0.0.0/0 0.0.0.0/0
3 ACCEPT All--0.0.0.0/0 0.0.0.0/0
4 ACCEPT TCP--0.0.0.0/0 0.0.0.0/0 state NEW TCP dpt:22
5 REJECT All--0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (Policy ACCEPT)
Num Target prot opt source destination
1 REJECT All--0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain OUTPUT (Policy ACCEPT)
Num Target prot opt source destination
[[Email protected] desktop]# service iptables stop
iptables:flushing firewall rules: [OK]
Iptables:setting chains to Policy Accept:filter [OK]
iptables:unloading modules: [OK]
[Email protected] desktop]# chkconfig iptables--list
Iptables0:off1:off 2:on3:on 4:on5:on 6:off
[Email protected] desktop]# chkconfig iptables off
[Email protected] desktop]# chkconfig iptables--list
Iptables0:off1:off 2:off3:off 4:off5:off 6:off
[Email protected] desktop]#
[[Email protected] desktop]# service iptables status
Iptables:firewall is not running.

Vii. SSH no key login (switch to Hadoop user)

Switch to Hadoop user
[email protected] ~]# su Hadoop
Generate public key and private key (3 prompts, always enter)
[[Email protected] root]$ CD
[Email protected] ~]$ pwd
/home/hadoop
[[email protected] ~]$ ssh-keygen-t RSA
Generating public/private RSA key pair.
Enter file in which to save the key (/HOME/HADOOP/.SSH/ID_RSA):
Enter passphrase (empty for no passphrase):
Enter same Passphrase again:
Your identification has been saved In/home/hadoop/.ssh/id_rsa.
Your public key has been saved in/home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
33:09:0b:6d:30:f5:07:10:40:0d:be:99:cf:a9:a4:92 [email protected]
The key ' s Randomart image is:
+--[RSA 2048]----+
|        . *=+o. |
| . +..       . |
| O +.      . |
| * o O |
| + . S |
| O. o |
| . . +          |
|           E O. |
| ..            . |
+-----------------+
[Email protected] ~]$
[[email protected] ~]$ ls
[Email protected] ~]$ ll-a
Total 36
DRWX------. 5 Hadoop hadoop 4096 Feb 28 14:19.
Drwxr-xr-x. 3 root root 4096 Feb 28 13:47.
-RW-------. 1 Hadoop hadoop 14:23. bash_history
-rw-r--r--. 1 Hadoop Hadoop bash_logout
-rw-r--r--. 1 Hadoop hadoop 176 Feb Bash_profile
-rw-r--r--. 1 Hadoop Hadoop 124 Feb BASHRC
Drwxr-xr-x. 2 Hadoop hadoop 4096 Nov gnome2
Drwxr-xr-x. 4 Hadoop hadoop 4096 Feb 06:11. Mozilla
DRWX------. 2 Hadoop hadoop 4096 Feb 14:23. SSH
[[Email protected] ~]$ CD. ssh/
[email protected]. ssh]$ ls
Id_rsa id_rsa.pub
[email protected]. ssh]$ LL
Total 8
-RW-------. 1 Hadoop hadoop 1671 Feb 14:23 Id_rsa
-rw-r--r--. 1 Hadoop hadoop 410 Feb 14:23 id_rsa.pub
[email protected]. ssh]$ CP id_rsa.pub Authorized_keys
[email protected]. ssh]$ LL
Total 12
-rw-r--r--. 1 Hadoop hadoop 410 Feb 14:26 Authorized_keys
-RW-------. 1 Hadoop hadoop 1671 Feb 14:23 Id_rsa
-rw-r--r--. 1 Hadoop hadoop 410 Feb 14:23 id_rsa.pub
[email protected]. ssh]$
SSH Login
[[email protected]. ssh]$ SSH Master
The authenticity of host ' master (192.168.209.100) ' can ' t be established.
RSA key fingerprint is f0:92:0b:08:0d:9b:72:0d:ca:99:30:0a:40:7e:05:ae.
SSH First login has this prompt, enter the good, then direct SSH master does not need the password to succeed
Is you sure want to continue connecting (yes/no)? Yes
warning:permanently added ' master,192.168.209.100 ' (RSA) to the list of known hosts.
[[Email protected] ~]$ exit
Logout
Connection to master closed.
[[email protected]. ssh]$ SSH Master
Last Login:fri Feb 14:27:32 from Master
[Email protected] ~]$
Exit SSH login for the next environment configuration
[[Email protected] ~]$ exit
Logout

Description: If this is a deployment of the Hadoop environment in Linux with the environment configuration, the final SSH configuration, later will introduce the principle analysis, and cluster multiple server configuration simple method.