Hadoop2.X/YARN environment setup-CentOS7.0 system configuration, centos7.0

Hadoop2.X/YARN environment setup-CentOS7.0 system configuration, centos7.0

I. Why should I choose CentOS7.0?

The official CentOS 7.0.1406 version was released at 17:39:42 on January 26, July 7. I used many Linux versions. For the environment configuration of Hadoop2.X/YARN, I chose CentOS7.0 for the following reasons:

1. The interface adopts the new GNOME interface of RHEL7.0, which is not comparable to CentOS6.5/RHEL6.5! (Of course, ora has adopted this style for a long time, but the current Fedora package shortage is no longer the same)

2. Once, I used RHEL7.0. The biggest problem with it was that YUM could not be used, and there was always a Warning prompt to register for purchase! To this end, you must modify the YUM source and modify other files so that you can use them without prompting them. After reading this, I think there will be a lot of kids shoes getting cooler!

3. Of course, I have used Ubuntu14.04/12.04 and elementaryOS for Ubuntu series. I don't know why, I feel ugly when I use Ubuntu14.04 !! However, many people say that CentOS and RHEL are ugly, but they do not know which is enterprise-level, and that is the overlord.

On the other hand, if visual effects are required, it is recommended to install elementaryOS. Of course, elementaryOS is also flawed: it has not provided updates for a long time and has been released for 12 years, it has not released new versions or updated websites for more than two years. At the same time, the Ubuntu model of elementaryOS has been updated for a long time. Therefore, many problems have been exposed, the trouble here is not to update the system at will. Otherwise, the black screen will be a pure command line. The beautiful system will suddenly be knocked back to the prototype, and the heart will be cool and cool, in addition, the updated apt-get won't be able to install some matching items, so elementaryOS is only for you!

4. Some people have used openSUSE, which is said to be the most gorgeous system. In my opinion, the fancy KDE shows a hop when opening a file/software, and my heart is itchy, what are you doing with a useless and ugly stuff ?? Why ?? In addition, the latest openSUSE is indeed unsatisfactory in Chinese design. In addition, the openSUSE environment is not very familiar, and the GNU/Linux design by Europeans is still difficult to adapt. Of course, I do not intend to change the flag to SUSE in Europe.

In fact, there are many GNU/Linux versions. It is enough to choose one suitable for you to continue playing and use it. There are many Linux versions and their design has their own merits, but the usage is similar, learn more about "differences! Of course, I would like to advise that beginners only need to run the virtual machine to see the effect, so they do not need to use each item! After all, experience is also one of the driving force of learning!

Ii. Experience the differences between CentOS7 and earlier versions"

Initial start

After the host is installed, the system interface is changed when the system is started. At first glance, it seems that there are two kernels. One is the rescue option, and after pressing e, all grub will be found. the conf information is complete, so it is hard to find the kernel to start, write a 1 (to enter the single user mode), and then enter the rescue mode.

Character Interface

After entering the system, it is a graphical interface. To enter the character interface, find the/etc/inittab and find that it is almost an empty file. If the file prompts you to change the runlevel, you can set/lib/systemd/system/runlevel *. target is soft connected to/etc/systemd/system/default.tar get (of course, You can overwrite it). After trying it, you can change it to runlevel3 on the Character interface.

Configure the network and Host Name

Then, if you want to configure the network, go to/etc/sysconfig/network-script/and check that the NIC has changed its name to enp1s5. This is interesting. After configuring the network, change the host name, change the name to/etc/sysconfig/network and find that the file is empty. Modify the file according to the previous version 6. It will not work after restart. man has a hostname, in the case of 7, you need to change the name to/etc/hostname.

Local yum source and Mount

Next, you want to configure a local yum source, and then you need to mount the CD, manually mount it, And then directly echo "mount ...... "To rc. local. After all, the script is started and executed. After the restart, the script is not mounted. Go to rc. in local, I saw that chmod + x rc should be manually executed in 7. local: Yes. You need to manually add the execution permission. Do you have to manually add the execution permission in the systems I installed before 6 !!!! Of course, after the permission is added, the mount command of rc. local is started after the instance is started.

LVM and xfs

Later, I wanted to dress up with an lvm for fun, so I tried partitioning and formatting (specially formatted as xfs), pv, vg, and lv step by step. It was okay, so I wanted to zoom in and out, when resize2fs is executed, how does one always report a superblock error? What is the problem? man resize2fs then finds that this command only supports ext file systems, then how is the installation of the system lvm? I have not solved this problem yet. There should be a solution.

Dhcp and services

Next, I want to dress a dhcp game. I was surprised to find that after packaging all dhcp, I couldn't start service dhcpd, and then found/etc/init. d/does not have a dhcp similar thing. After all, there were dhcpd, dhcpd6, and dhcrelay before 6, and then we found that/sbin/dhcpd had a STARTUP file. Isn't it, if you want to enable services in the future, do you have to manually compile them? Wrong, it is in/usr/lib/systemd/system/dhcpd. service, there are still many modifications, and then add permissions to execute service restart/stop ...... Yes, but it points to the systemctl restart/start/stop service. service

Iptables

I also found that iptables is not a service in/etc/init. d/this time, but under/sbin;

Firewall changed from iptables to firewalld

 

Iii. Disable firewall in System Configuration

Systemctl stop firewalld. service # stop firewall

Systemctl disable firewalld. service # disable firewall startup

The basic operating format of the systemctl command is:

Systemctl Action service name. service
Action: start, stop, restart, status, enable, disable, is-enabled

4. Configure the Linux Network

1. Modify the hostname

Modify the hostname of CentOS 7.0 with the root user in/etc/hostname:

[Root @ hadoop1 ~] # Vim/etc/hostname # edit/etc/hostname
[Root @ hadoop1 ~] # Cat/etc/hostname # view/etc/hostname
Hadoop1
[Root @ hadoop1 ~] #

In earlier CentOS versions, modify the following with the root user in/etc/sysconfig/network:

[root@hadoop1 ~]# vim /etcsysconfig/network

The configuration items of "/etc/sysconfig/network" are as follows:

Whether NETWORKING uses the network
The default GATEWAY of the GATEWAY is consistent with the default GATEWAY in the property file of the configured IP address.
IPGATEWAYDEV Default Gateway Interface Name
HOSTNAME Host Name
DOMAIN Name

Hostname takes effect after reboot

2. Modify the IP address of the Current Machine

Step 1: Enter the configuration file directory

Now we have the root permission. We cd to the directory of the network configuration file and list the files under the directory.

Step 2: modify the configuration file

Under this directory, the file named "ifcfg-ens192" is the master file of the network configuration. Vi it!

Modify or add:

ONBOOT = yes
BOOTPROTO = static
IPADDR = IP Address
NETMASK = 255.255.255.0 # Subnet Mask
GATEWAY = 192.168.30.1 # GATEWAY address

Step 3: restart the network service

After the configuration file is modified,: wq! Save and exit! Then restart the network.

Step 4: Test

Run the ifconfig command to check whether the ip address is successfully set and ping the network. This time should be successful!

 

3. Modify DNS (depending on the situation, optional)

If the network environment has DNS requirements, configure it. Otherwise, do not configure it!

Format:

Nameserver DNS address

[Root @ hadoop1 ~] # Vi/etc/resolv. conf nameserver 202.131.80.1 # nameserver 202.131.80.5 as required # as required

4. Configure the hosts file

The "/etc/hosts" file is used to configure the DNS server information to be used by the host. It records the corresponding [HostName and IP address] of each host in the LAN. When you are connecting to the network, first find the file and find the IP address corresponding to the Host Name (or domain name.

We want to test whether the two machines can communicate with each other. Generally, we use "ping the IP address of the machine". If we want to use "ping the Host Name of the machine" to find the machine with this name, the solution is to modify the "/etc/hosts" file and solve the problem by writing the IP addresses and hostnames of hosts in the LAN to the file one by one.

For example, if the machine is "hadoop1: 59.67.107.80", use the command "ping" to test the connection. The test results are as follows:

From the value in, directly test the IP address, can ping, but test the host name, if not ping, the prompt "unknown host -- unknown host ", view the content of the "/etc/hosts" file of "hadoop1.

If the ping hadoop1 rule is not general, the following information is displayed:

If the general rules of ping hadoop1 are as follows:

If "59.67.107.79 hadoop2" is not found, the host name cannot be parsed as "hadoop2.

In Hadoop cluster configuration, you need to add the IP address and host name of all machines in the cluster in the "/etc/hosts" file, in this way, the Master can communicate with all Slave machines not only through IP addresses, but also through host names. Therefore, add the following content to the end of the "/etc/hosts" file on all machines:

Because this article describes the configured environment, my/etc/hosts displays:

Generally, the instance is in the Intranet. Add it as follows (192.168.1.1 is the gateway ):

192.168.1.2 hadoop1192.168.1.3 hadoop2192.168.1.4 hadoop3192.168.1.5 hadoop4

Run the following command:

vim /etc/hosts

Now we are conducting a ping test on the host name of the machine "hadoop2" to check whether the test is successful.

From this, we can use the host name for ping, indicating that the content we just added can be resolved in the LAN, now, the remaining thing is to perform the same configuration on the remaining Slave machines. Then perform the test.

How should we build a complete environment for android development? What is required for specific environment configuration?

Eclipse3.5 or 3.6 + ADT + JDK1.5 or 1.6 + android-sdk06 (07)-windows
For detailed steps, see www.javaeye.com/wiki/topic/444988

Hello, how do you configure local win7 in Eclipse to access hadoop220 under Centos?

For code access, you only need to configure the fs. default. name attribute. If you are using Eclipse to integrate the mapreduce plug-in, the main problem is the Eclispe version. You can either find a version online or try it on your own.