Hadoop cluster installation and configuration + DNS + NFS in the production environment

The production environment of Hadoop cluster installation and configuration + DNS + NFS environment LinuxISO: CentOS-6.0-i386-bin-DVD.iso32 bit JDKversion: 1.6.0 _ 25-eaforlinuxHad ..

The production environment of Hadoop cluster installation and configuration + DNS + NFS environment LinuxISO: CentOS-6.0-i386-bin-DVD.iso32 bit JDKversion: 1.6.0 _ 25-eaforlinuxHad ..

Hadoop cluster installation and configuration + DNS + NFS in the production environment

I. Environment

Linux ISO: CentOS-6.0-i386-bin-DVD.iso 32-bit

JDK version: "1.6.0 _ 25-ea" for linux

Hadoop software version: hadoop-0.20.205.0.tar.gz for linux

VMware®Workstation Version: 7.0.0 build-203739

My linux Virtual Machine configures master slave1 slave2 as follows

Host Name

IP

Node name

Remarks

H1

192.168.2.102

Master

Namenode and jobtracker

H2

192.168.2.103

Slave1

Datanode and tasktracker

H4

192.168.2.105

Slave2

Datanode and tasktracker

Thank you for choosing yanggw8071 mikeliu chizeng.

1.

2.

3.

2. Bind the DNS server and resolve the Host Name

1. I have installed the DNS server on the h1 node (master) and then parsed the Host Name of the h1 h2 h4 node.

2. the h1 node is required to access the Internet, because we need to install the DNS rpm package in yum mode. For us space, the Internet access method is as follows:

CentOS needs to be connected to the network. For more information, see the NOSQL series-memcached Installation Management and repcached high availability. This article first introduced how to connect a virtual machine to the Internet.

3. We used the hosts file to parse the host name. Now we can delete it without using it. I will change the name directly, Hong Kong server, so we can keep the backup.

[Root @ h1 etc] # mv/etc/hosts/etc/hosts_20130126

4. some packages use the webmin GUI to configure the DNS server. This method is user-friendly on the interface, but I am used to the command line) therefore, you can directly use rpm to install DNS.

5. Check whether the bind package exists. This also uses the chroot mechanism.

[Root @ h1 ~] # Rpm-q bind

Package bind is not installed

[Root @ h1 ~] # Rpm-q bind-chroot

Package bind-chroot is not installed

It is shown that no installation is available, so we need to install it in yum mode.

6. Now use yum to install bind and bind-chroot packages

[Root @ h1 ~] # Yum-y install bind-chroot

Omitted ......

Dependencies Resolved

========================================================== ========================================================== ========

Package Arch Version Repository Size

========================================================== ========================================================== ========

Installing: install these two packages

Bind i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 updates 4.0 M

Bind-chroot i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 updates 70 k

Updating for dependencies: upgrade two dependent packages

Bind-libs i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 updates 882 k

Bind-utils i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 updates 180 k

Transaction Summary

========================================================== ========================================================== ======= Install 2 Package (s)

Upgrade 2 Package (s)

Installed:

Bind. i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 bind-chroot.i686 32: 9.8.2-0.10.rc1.el6 _ 3.6

Dependency Updated:

Bind-libs.i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 bind-utils.i686 32: 9.8.2-0.10.rc1.el6 _ 3.6

Complete!

The bind and bind-chroot packages have been installed.

[Root @ h1 ~] # Rpm-q bind

Bind-9.8.2-0.10.rc1.el6_3.6.i686

[Root @ h1 ~] # Rpm-q bind-chroot

Bind-chroot-9.8.2-0.10.rc1.el6_3.6.i686

Haha now all have been installed

7. There are three packages to install in yum install bind *.

[Root @ h1 etc] # yum install bind *

Dependencies Resolved

========================================================== ========================================================== ========

Package Arch Version Repository Size

========================================================== ========================================================== ========

Installing: install these three packages

Bind-devel i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 updates 376 k

Bind-dyndb-ldap i686 1.1.0-0.9.b1.el6 _ 3.1 updates 63 k

Bind-sdb i686 32: 9.8.2-0.10.rc1.el6 _ 3.6 updates 305 k

Transaction Summary

========================================================== ========================================================== ======= I

Install 3 Package (s)

Upgrade 0 Package (s)

Installed:

Bind-devel.i686 32: 9.8.2-0.10.rc1.el6-3.6

Bind-dyndb-ldap.i686. 1.0-0.9.b1.el6 _ 3.1

Bind-sdb.i686 32: 9.8.2-0.10.rc1.el6-3.6

Complete!

The installation is complete.

8. Modify the named. conf file, directory/etc/named. conf

[Root @ h1 etc] # vim/etc/named. conf is marked with red

Options {

Listen-on port 53 {any ;}; // change 127.0.0.1 to any

Listen-on-v6 port 53 {: 1 ;};

Directory "/var/named ";

Dump-file "/var/named/data/cache_dump.db ";

Statistics-file "/var/named/data/named_stats.txt ";

Memstatistics-file "/var/named/data/named_mem_stats.txt ";

Allow-query {any ;}; // modify localhost to any

Recursion yes;

Dnssec-enable yes;

Dnssec-validation yes;

Dnssec-lookaside auto;

/* Path to isc dlv key */

Bindkeys-file "/etc/named. iscdlv. key ";

Managed-keys-directory "/var/named/dynamic ";

};

Logging {

Channel default_debug {

File "data/named. run ";

Severity dynamic;

};

};

Zone "." IN {

Type hint;

File "named. ca ";

};

Include "/etc/named. rfc1912.zones ";

Include "/etc/named. root. key ";

9. Modify the named. rfc1912.zones file, directory/etc/named. rfc1912.zones

Configure forward and reverse configuration files

[Root @ h1 etc] # vim/etc/named. rfc1912.zones Add the following content at the end of this file. Red is the place to note.

Zone "leonarding.com" IN {

Type master;

File "leonarding.com. zone"; note that this is your own domain name, remember

Allow-update {none ;};

};

Zone "Limit 192.in-addr. arpa" IN {

Type master;

File "2.168.192.in-addr. zone"; this is the reverse configuration file.

Allow-update {none ;};

};

10. Create the leonarding.com. zone and ipv192.in-addr. zon files.

[Root @ h1 etc] # enter the cd/var/named directory

[Root @ h1 named] # cp-p named. localhost leonarding.com. zone

[Root @ h1 named] # cp-p named. localhost created 192.in-addr. zone

Modify these two files to a new file. These two files are regional files. Make sure they are correct; otherwise, an error is reported.

Add forward file and configure

[Root @ h1 named] # vim leonarding.com. zone

$ TTL 86400

@ In soa h1.leonarding.com. chizk .(

0; serial (d. adams)

1D; refresh

1 H; retry

1 W; expiry

3 H); minimum

@ In ns h1.leonarding.com.

H1.leonarding.com. in a 192.168.2.102

H2.leonarding.com. in a 192.168.2.103

H4.leonarding.com. in a 192.168.2.105

Add reverse file and configure

[Root @ h1 named] # vim rule 192.in-addr. zone

$ TTL 86400

@ In soa h1.leonarding.com. chizk .(

0; serial (d. adams)

1D; refresh

1 H; retry

1 W; expiry

3 H); minimum

@ In ns h1.leonarding.com.

102 in ptr h1.leonarding.com.

103 in ptr h2.leonarding.com.

105 in ptr h4.leonarding.com.

Modify the group of forward and reverse files, and use chgrp.

[Root @ h1 named] # chgrp named leonarding.com. zone

[Root @ h1 named] # chgrp named leonarding.com. rev

11. Modify/etc/resolv. conf to add the ip address of the DNS domain name server.

H1 (master)

[Root @ h1 named] # vim/etc/resolv. conf

Nameserver 192.168.2.102 can be appended to the end, that is, the ip address of my master machine.

This means that the master machine is used as the DNS domain name resolution server,

H2 (slave)

[Root @ h2 sysconfig] # vim/etc/resolv. conf

Nameserver 192.168.2.102

H4 (slave)

[Root @ h4. ssh] # vim/etc/resolv. conf

Nameserver 192.168.2.102

Add the DNS Domain Name Server ip address. When domain name or host name resolution is available, you will know which machine to map.

12. Start the DNS server

[Root @ h1 named] # service named start

Start named: [OK]

13. Use the nslookup command to test domain name resolution

Disable firewall before testing

[Root @ h1 named] # service iptables status

Iptables: Firewall not running

Forward DNS-> IP

[Root @ h1 named] # nslookup h1.leonarding.com

Server: 192.168.2.102 indicates the IP address of the Domain Name Server.

Address: 192.168.2.102 #53 ip + port number, default DNS port 53

Name: h1.leonarding.com domain Name resolution

Address: 192.168.2.102 ing ip Address

[Root @ h1 named] # nslookup h2.leonarding.com

Server: 192.168.2.102 indicates the IP address of the Domain Name Server.

Address: 192.168.2.102 #53 ip + port number, default DNS port 53

Name: h2.leonarding.com domain Name resolution

Address: 192.168.2.103 ing ip Address

[Root @ h1 named] # nslookup h4.leonarding.com

Server: 192.168.2.102 indicates the IP address of the Domain Name Server.

Address: 192.168.2.102 #53 ip + port number, default DNS port 53

Name: h4.leonarding.com domain Name resolution

Address: 192.168.2.105 ing ip Address

Reverse resolution IP-> Domain Name

[Root @ h1 named] # nslookup 192.168.2.102

Server: 192.168.2.102

Address: 192.168.2.102 #5

102.20.192.in-addr. arpa name = h1.leonarding.com.

[Root @ h1 named] # nslookup 192.168.2.103

Server: 192.168.2.102

Address: 192.168.2.102 #53

103.20.192.in-addr. arpa name = h2.leonarding.com.

[Root @ h1 named] # nslookup 192.168.2.105

Server: 192.168.2.102

Address: 192.168.2.102 #53

105.20.192.in-addr. arpa name = h4.leonarding.com.

We can also use the host command for parsing and testing.

[Root @ h1 named] # host h2.leonarding.com

H2.leonarding.com has address 192.168.2.103

[Root @ h1 named] # host 192.168.2.103

103.20.192.in-addr. arpa domain name pointer h2.leonarding.com.