hadoop2.4.1 Pseudo-distributed construction

Source: Internet
Author: User

1. Preparing the Linux environment
1 Click the VMware shortcut, right-click the file location, double-clicking Vmnetcfg.exe, VMnet1 host-only, modify the subnet IP Settings network segment: 192.168.244.131. Subnet mask: 255.255.255.0, apply, OK
Back to Windows--open network and Sharing Center, change adapter settings, right-VMnet1 properties, double-click IPv4, set Windows ip:192.168.244.1 Subnet mask: 255.255 .255.0 Click OK
On the virtual software--my computer--------right-----Settings network adapter,
1.1 Modifying host Names
Vim/etc/sysconfig/network

Networking=yes
Hostname=master # # #

Hostname Master


1.2 Modifying IP
Two different ways:
First: Modify with the Linux GUI (highly recommended)
Go to the Linux GUI-right click on the two small computers in the upper right-click Edit Connections, select the current network system eth0, click the Edit button, and select the option IPv4 Add ip:192.168.8.118 subnet Mask by clicking the Add button, Manual: 255.255.255.0 gateway: 192.168.1.1, apply

The second kind: Modify the configuration file way (The Cock Silk program ape dedicated)
Vim/etc/sysconfig/network-scripts/ifcfg-eth0

Device= "Eth0"
Bootproto= "Static" # # #
Hwaddr= "00:0c:29:3c:bf:e7"
ipv6init= "Yes"
Nm_controlled= "Yes"
onboot= "Yes"
Type= "Ethernet"
Uuid= "ce22eeca-ecde-4536-8cc2-ef0dc36d4a8c"
Ipaddr= "192.168.8.118" # # #
Netmask= "255.255.255.0" # # #
Gateway= "192.168.8.1" # # #

1.3 Modifying the mappings between host names and IPs
Vim/etc/hosts

192.168.8.118master

CTRL + C stops the ping execution.


1.4 Shutting down the firewall
#查看防火墙状态
Service Iptables Status
#关闭防火墙
Service Iptables Stop
#查看防火墙开机启动状态
Chkconfig iptables--list

Chkconfig--list | grep iptables
#关闭防火墙开机启动
Chkconfig iptables off

1.5 Restarting Linux
Reboot

2. Installing the JDK

1.7 Installing the JDK
Execution commands (1) cd/usr/local
(2) chmod u+x Jdk-6u24-linux-i586.bin
(3)./jdk-6u24-linux-i586.bin
(4) MV Jdk-1.6.0_24 JDK
(5) The contents of Vi/etc/profile additions are as follows:
Export JAVA_HOME=/USR/LOCAL/JDK
Export path=.: $JAVA _home/bin: $PATH
(6) Source/etc/profile
Verification: Java-version

Extended:

Delete Folder instance:
rm-rf/var/log/httpd/access
will delete/var/log/httpd/ Access directory and all files under it, folder

2.2 extract JDK
#创建文件夹
Mkdir/usr/java
#解压
TAR-ZXVF Jdk-7u55-linux-i586.tar.gz-c/usr/java/

2.3 Add Java to environment variables
vim/etc/profile
#在文件最后添加
Export java_home=/usr/java/jdk
Export path= $PATH: $JAVA _home/bin

#刷新配置
source/etc/ Profile

3. Install hadoop2.4.1

Execution commands (1) TAR-XZVF hadoop-1.1.2.tar.gz
(2) MV hadoop-1.1.2 Hadoop
(3) The contents of Vi/etc/profile additions are as follows:
Export JAVA_HOME=/USR/LOCAL/JDK
Export Hadoop_home=/usr/local/hadoop
Export path=.: $HADOOP _home/bin: $JAVA _home/bin: $PATH
(4) Source/etc/profile
(5) Modify the configuration files under the Conf directory hadoop-env.sh, Core-site.xml, Hdfs-site.xml, Mapred-site.xml
(6) Hadoop Namenode-format
(7) start-all.sh
Verification: (1) Execute command JPS if you see 5 new Java processes, namely Namenode, Secondarynamenode, DataNode, Jobtracker, Tasktracker
(2) View in browser, http://chaoren:50070 http://chaoren:50030


Note: The hadoop2.x configuration file $hadoop_home/etc/hadoop
Pseudo-distributed requires 5 configuration files to be modified
3.1 Configuring Hadoop
First one: hadoop-env.sh
Vim hadoop-env.sh
#第27行
Export java_home=/usr/java/jdk1.7.0_65

The second one: Core-site.xml
<!--the address of the boss of HDFs (NameNode)--
<property>
<name>fs.defaultFS</name>
<value>hdfs://itcast01:9000</value>
</property>
<!--specify the storage directory where the Hadoop runtime generates files--
<property>
<name>hadoop.tmp.dir</name>
<value>/itcast/hadoop-2.4.1/tmp</value>
</property>

The third one: Hdfs-site.xml
<!--Specify the number of HDFs replicas-
<property>
<name>dfs.replication</name>
<value>1</value>
</property>

Fourth: Mapred-site.xml (MV Mapred-site.xml.template mapred-site.xml)
MV Mapred-site.xml.template Mapred-site.xml
Vim Mapred-site.xml
<!--specify Mr to run on yarn--
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

Fifth one: Yarn-site.xml
<!--Specify the address of yarn's boss (ResourceManager)---
<property>
<name>yarn.resourcemanager.hostname</name>
<value>itcast01</value>
</property>
<!--Reducer How to get data--
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>

3.2 Adding Hadoop to an environment variable

Vim/etc/proflie
Export java_home=/usr/java/jdk1.7.0_65
Export hadoop_home=/itcast/hadoop-2.4.1
Export path= $PATH: $JAVA _home/bin: $HADOOP _home/bin: $HADOOP _home/sbin

Source/etc/profile

3.3 Formatting Namenode (initialization of Namenode)
HDFs Namenode-format (Hadoop namenode-format)

3.4 Starting Hadoop
Start HDFs First
sbin/start-dfs.sh

Start yarn Again
sbin/start-yarn.sh

3.5 Verifying whether the startup was successful
Using the JPS command to verify
27408 NameNode
28218 Jps
27643 Secondarynamenode
28066 NodeManager
27803 ResourceManager
27512 DataNode

http://192.168.8.118:50070 (HDFs management interface)
http://192.168.8.118:8088 (Mr Management interface)

4. Configure SSH Free Login
#生成ssh免登陆密钥
#进入到我的home目录
CD ~/.ssh

Ssh-keygen-t RSA (four return)
After executing this command, two files Id_rsa (private key), id_rsa.pub (public key) will be generated
Copy the public key to the machine that you want to avoid landing on
Ssh-copy-id localhost

"Dark Horse programmer HADOOP7 Day Video tutorial full network debut"
More tutorials at the following shops:

Address 1:http://chinajava.taobao.com
Address 2:http://shop111670867.taobao.com
Address 3:http://shop113561352.taobao.com
Address 4:http://shop113561276.taobao.com

If you can't contact us, please add qq:1871209564.

hadoop2.4.1 Pseudo-distributed construction

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.