Build a pseudo-distributed hadoop Environment
1. network connection between the host machine (Windows) and the client (Linux installed in a virtual machine.
A) The host-only host is connected to the client separately;
Benefits: Network isolation;
Disadvantage: the virtual machine cannot communicate with other servers;
B. The bridge host is in the same LAN as the client.
Benefit: the sinus can access each other in the same LAN;
Disadvantage: incomplete.
2. Distributed installation steps for hadoop
A) set static IP addresses
Right-click the icon in the upper-right corner of the lower-left corner of centos to modify it;
Restart the NIC service network restart;
Verify: Execute the command ifconfig
B) modify the Host Name (Original Name: casichadoop) hostname to view the current name
Modify the current Session host name: Set the name hostname hadoop
In the configuration file, set hostname VI/etc/sysconfig/network to hadoop.
C) bind the hostname and IP address
Run the command VI/etc/hosts to add 10.10.10.10.
Save and exit
Verification: Ping hadoop
D. Disable the firewall and view the service iptables status first.
Service iptables stop
E) Check chkconfig -- list | grep iptables when the firewall is automatically started.
Disable Automatic Start firewall chkconfig iptables off
Verification: chkconfig -- list | grep iptables
F) SSH (Secure Shell) password-free Login
<1> execute the command Shh-keygen-t rsa to generate the key, which is located in ~ /. SSH
<2> Execute Command CP ~ /. Ssh/id_rsa.pub ~ /. Ssh/authorized_keys
Verification: SSH localhost
G) install JDK
L open the directory CD/usr/local
L then delete all files Rm-RF *
L copy all files to the current directory CP/root/downloads /*.
L view the permissions of the current directory LS-l
L grant permission to execute chmod U + x jdk-6u24-linux-i586.bin
L execute JDK installation./jdk-6u24-linux-i586.bin
L mv jdk1.6.0 _ 24 JDK rename
L set the environment variable VI/etc/profile
N export java_home =/usr/local/JDK
N export Path =./$ java_home/bin: $ path
N source/etc/profile
N verification: Java-version
H) install hadoop
Hadoop-1.1.2.tar.zip I. Tar-zxvf
Ii. Mv hadoop-1.1.2 hadoop
Iii. Set Environment Variables
N export hadoop_home =/usr/local/hadoop
N export Path =.: $ hadoop_home/bin: $ java_home/bin: $ path
Iv. Modify four configuration files
1. Modify hadoop-env.sh
Export java_home =/usr/local/JDK
2. Modify core-site.xml
<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration><property><name>fs.default.name</name><value>hdfs://hadoop:9000</value><description>hadoop</description></property><property><name>hadoop.tmp.dir</name><value>/usr/local/hadoop/tmp</value></property></configuration>
3. Modify hdfs-site.xml
<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration><property><name>dfs.replication</name><value>1</value></property><property> <name>dfs.permissions</name><value>false</value></property></configuration>
4. Modify mapred-site.xml
<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration><property><name>mapred.job.tracker</name><value>hadoop:9001</value><description>change you own hostname</description></property></configuration>
V. Run the hadoop namenoe-format command to format HDFS
Vi. Execute Command start-all.sh
VII. Execute the command JPs to view the process
Hadoop learning notes: hadoop pseudo-Distributed Environment Construction