Hadoop2.x/yarn Environment Build--centos7.0 System Configuration _ database Other

One, why I choose CentOS7.0

July 7, 14 17:39:42 released the official version of CentOS 7.0.1406, I have used a variety of Linux, for the hadoop2.x/yarn of the environmental configuration to choose why CentOS7.0, the reasons are:

1, the interface using RHEL7.0 new GNOME interface Wind, this is not centos6.5/rhel6.5 can compare! (Of course, Fedora used this style long ago, but now the fedora is not the case of the package)

2, once, I also used RHEL7.0, it is the biggest problem is Yum can not use, and there will always be warning prompt registration purchase! To this end, must modify the Yum source, and modify other documents so as not to use, see so troublesome, presumably have a lot of child shoes drop heart is already cool 4/7!

3, of course, Ubuntu Series I used ubuntu14.04/12.04 and elementaryos, I do not know why, a Ubuntu14.04 feel so ugly!! But a lot of people said CentOS and Rhel more ugly, but no one knows which is the enterprise class, that is the Overlord.

On the other hand, the need for visual effects, suggest to install a elementaryos to try, of course, Elementaryos also has a flaw: it has been a long time no update, launched in 12, has not released a new version of the system, also did not update the site, at the same time, Elementaryos model Ubuntu has been replaced, so, exposed a lot of problems, which is the trouble is not to update the system, otherwise black screen, that is really pure command line, beautiful system was suddenly hit back to the prototype, the heart is definitely scar cool scar, And it is updated after the Apt-get is unable to install some matching some, so elementaryos only for the enjoyment of playing!

4, has also been used openSUSE, is said to be the most gorgeous system, in my opinion, the fancy KDE open file/software show a jump, I was itching, all right son you get a useless and ugly dongdong why?? What are you doing? In addition, the latest openSUSE to Chinese design is indeed unsatisfactory. In addition, the openSUSE environment is not familiar, the European design of the gnu/linux still some difficult to adapt, of course, I do not intend to change the flag banner to Europe SuSE defection.

In fact, gnu/linux version of many, choose a suitable for their one has been playing, use it enough, Linux version of a large number of different designs, but use the same, understand the "different" point can! Of course, or to advise a: Beginners only run virtual machine to see the effect is good, MO will use each of the will! After all, experience is also one of the driving force of learning!

Second, experience CentOS7 and previous version of the "Different"

Initial start

After loading the machine, boot into the system interface changed, at first glance thought to be two cores, originally has one is rescue the choice, moreover presses E, will discover all grub.conf information all out, finally found the kernel to start the place, wrote a 1 (to enter Single-user mode), And then into the rescue mode.

Character interface

Into the system, is the graphical interface, want to enter the character interface, the results found/etc/inittab, found almost an empty file, the document prompts want to change runlevel, you can/lib/systemd/system/runlevel*.target soft connect to/etc/ Systemd/system/default.target under (of course, cover also line), tried a while, can be changed to the character interface Runlevel3.

Configuring Network and host names

Then, want to configure the network, into the/etc/sysconfig/network-script/a look, I, the network card renamed ENP1S5, good fun, configure the network after the change of host name, to/etc/sysconfig/network to change the name, Found that the file is also empty, according to the original version 6 to modify, no effect after reboot, man a bit hostname, found 7 renamed to/etc/hostname to change the name.

Local Yum Source and Mount

Next, you want to configure a local Yum source, and then hang the disc, manually hang up, and then directly echo "Mount ..." Into the rc.local, after all, is to start the execution script, restart after unexpectedly did not hang, go in rc.local see a moment, 7 unexpectedly want to manually put chmod +x rc.local, yes, to manually add executive permissions, I installed before 6 of the system before this will have to manually enforce permissions, I don't remember,!!!!. Of course, after the power is added, the boot execution of the rc.local Mount command.

LVM and XFS

Later, want to dress an LVM play, so ah on the partition, format (deliberately formatted for XFS), pv,vg,lv step by step, are no problem, and then think of magnifying, narrowing, then the problem, the implementation of RESIZE2FS, how has been reported superblock or something wrong, What's going on, then man Resize2fs, found that this command only supports EXT file system, I 艹, that I installed the system is how LVM, this problem I have not resolved, there should be a solution.

DHCP and services

Then, want to dress a DHCP play, surprised to find all the packaging on the DHCP, can not service dhcpd start, and then found that/etc/init.d/did not even have a DHCP similar things, after all, 6 before the DHCPD,DHCPD6, Dhcrelay, and then found that/SBIN/DHCPD have a startup file, do you think, after the service to start, you have to manually edit? Wrong, is in/usr/lib/systemd/system/dhcpd.service, also want to modify a lot, and then add permissions, perform service services restart/stop ... OK, but point to systemctl restart/start/stop services. Service

Iptables

Also found iptables this time is not as a service under the/etc/init.d/,/sbin under;

The firewall was replaced by iptables into FIREWALLD.

Third, the system configuration of the shutdown firewall

Systemctl Stop Firewalld.service #停止firewall

Systemctl Disable Firewalld.service #禁止firewall开机启动

The basic operational format for the SYSTEMCTL command is:

Systemctl Action Service name.
Action: start,stop,restart,status,enable,disable,is-enabled

Iv. Configuring Linux Networks

1. Modify hostname

The hostname of CentOS 7.0 is modified in/etc/hostname with the root user:

[Root@hadoop1 ~]# vim/etc/hostname #编辑/etc/hostname
[Root@hadoop1 ~]# cat/etc/hostname #查看/etc/hostname
Hadoop1
[Root@hadoop1 ~]#

Previous versions of CentOS were modified with the root user in/etc/sysconfig/network:

[Root@hadoop1 ~]# Vim/etcsysconfig/network

The "/etc/sysconfig/network" project is set out below:

Networking whether to use the network
The gateway default gateway is consistent with the default gateway in the properties file that is then configured for IP
Ipgatewaydev the interface name of the default gateway
HOSTNAME Host Name
Domain Name

Reboot after hostname effective

2, modify the current machine IP

STEP1: Access to the configuration file directory

Now that we have root permissions, we CD to the directory of the network profile and list the files under the directory.

STEP2: Modifying configuration files

In this directory, the filename "ifcfg-ens192" file is the primary file for the network configuration. VI It!

Modify or add:

gateway=192.168.30.1 #网关地址

STEP3: Restart network services

After modifying the configuration file,: wq! Save exit! Then restart the network.

STEP4: Testing

Use the Ifconfig command to see if IP is set to succeed, and then ping the network. This time should be successful!

3, modify DNS (see situation, not required to choose)

If the network environment has DNS requirements, then configure, otherwise, do not match!

Format:

NameServer DNS Address

[Root@hadoop1 ~]# vi/etc/resolv.conf 
nameserver 202.131.80.1  #依需求而定 nameserver 202.131.80.5  #依需求而定

4, configure the Hosts file

The "/etc/hosts" file is used to configure the DNS server information that the host will use, and it is used to document the corresponding [hostname and IP] of the successive hosts in the LAN. When a user is making a network connection, the file is first searched for the corresponding IP address of the host name (or domain name).

We want to test whether the two machines are connected, generally use "Ping machine's IP", if you want to use "Ping machine hostname" found that the name of the machine, the solution is to modify the "/etc/hosts" this file, The problem can be solved by writing the IP address of each host in the LAN to the file with the hostname one by one correspondence.

For example: The machine for "hadoop1:59.67.107.80" to the Machine for "hadoop2:59.67.107.79" with the command "ping" memory connection test. The test results are as follows:

From the value above, directly to the IP address test, can ping, but the host name test, if not ping, prompted "Unknown host--unknown host", then View "HADOOP1"/etc/hosts file contents.

If Ping Hadoop1 does not show the general rule:

If ping Hadoop1 general rules show:

If found inside there is no "59.67.107.79 hadoop2" content, so this machine is unable to the machine's host name "HADOOP2" resolution.

In the Hadoop cluster configuration, the IP and hostname of all the machines in the cluster need to be added to the "/etc/hosts" file so that master and all slave machines can communicate not only over IP but also through host names. So add the following to the end of the "/etc/hosts" file on all machines:

Because this article is already configured with a description of the environment, my/etc/hosts shows:

Generally under the intranet, according to this add (192.168.1.1 for the Gateway):

192.168.1.2 hadoop1
192.168.1.3 hadoop2
192.168.1.4 hadoop3 192.168.1.5 hadoop4

Add with the following command:

Vim/etc/hosts

Now we are doing a ping test on the hostname of the machine for "HADOOP2" to see if it can be tested successfully.

From the above we have been able to ping the host name to explain what we have just added, in the LAN can be DNS resolution, then the remaining thing is the rest of the slave machine on the same configuration. Then test.