Environment Building-centos cluster building

Environment Building-centos cluster building

Preface
Recently, many friends asked me how to build a hadoop distributed cluster of big data. Therefore, write a blog to help more people who are just getting started with big data. This blog will show you step by step how to build a hadoop. Follow the steps on the blog. You will soon have your own hadoop cluster.

I use virtualbox on my windows machine to virtualize three machines as an experiment environment.

Lab Environment
Windows Host: Windows 10 (the rest are acceptable)
Virtualbox version: virtualbox 5.1.22
Centos version: centos 6.9.
Xshell version: xshell 5

It may be difficult for many friends to find software. Here I share my link to Baidu cloud for you to download:
Link: http://pan.baidu.com/s/1c24gbUK password: 8h1r

Next, we will start creating virtual machines.

Use virtualbox to create a VM

Use virtualbox to create three centos as the project lab cluster.

Follow these steps:

Open the virtualbox software. We have a new one:

Next, click set.

Set our storage and add our images

Set our network connection (important)

After the preceding settings are complete, we can enable and start to install centos.

The following two items are the default values. Pay attention to them here.

Set the time zone to Asia and Shanghai

Set the password. For convenience, set it to 123456.

Because the password security is too low, you can choose to use it anyway.

Disk Partition. The default value is good.

Wait for it to be installed on its own. At this time, you can follow this step to create two new machines.

The reboot button appears. Restart the instance.

After the restart, enter the account password and enter:

At the beginning, there was no IP address, and we needed to set it,

Set IP

Set the network adapter onboot = yes.

[[Email protected] ~] # Cd/etc/sysconfig/network-scripts/[[email protected] network-Scripts] # [[email protected] network-Scripts] # vi ifcfg-eth0 device = eth0hwaddr = 08: 00: 27: 72: EC: eetype = ethernetuuid = 6b262313-ddd5-4a97-bb75-771ea5621f91onboot = yesnm_controlled = yesbootproto = DHCP #: WQ save and quit

Restart the network

[[email protected] network-scripts]# service network restart

Test, Ping Baidu:

Here we have installed centos. It indicates that the network must be connected. Otherwise, it will be troublesome in the future.
Follow these steps to create two new machines.

Use xshell to connect to our server

The xshell software is still very useful,

New Connection

Next, enter the user and password.
Select the box below so that you do not need to enter the account password.

The above indicates that the logon is successful.

Install Common Software

The following installation of the lrzsz software is used to transfer files. It is a practical software. Of course, you can also choose your own method.

[[email protected] ~]# yum -y install lrzsz....Installed:  lrzsz.x86_64 0:0.12.20-27.1.el6                                                       Complete!

Okay. Now we have installed it.

Install the JDK Environment

Before installation,
First, we should create a new directory, which is more standardized. This is just a matter of personal habits.

[[Email protected] ~] # [Email protected] ~] # Mkdir app [[email protected] ~] # Mkdir softwares [[email protected] ~] # [Email protected] ~] # Mkdir lib [[email protected] ~] # [Email protected] ~] # Mkdir data [[email protected] ~] # App installation directory # softwares software storage directory # lib package storage directory # data test data storage directory

Next, we will upload the JDK file to our server,
Use the RZ command

After the upload is complete, decompress the package to the app directory.

[[email protected] softwares]# tar -zxvf jdk-8u144-linux-x64.tar.gz -C ../app/

After decompression, you can configure environment variables.

[[email protected] ~]# vim ~/.bash_profile # .bash_profile# Get the aliases and functionsif [ -f ~/.bashrc ]; then        . ~/.bashrcfiexport JAVA_HOME=/root/app/jdk1.8.0_144export PATH=$JAVA_HOME/bin:$PATH

Next, let's make our environment variables take effect,

[[email protected] ~]# source ~/.bash_profile[[email protected] ~]# [[email protected] ~]# java -versionjava version "1.8.0_144"Java(TM) SE Runtime Environment (build 1.8.0_144-b01)Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)[[email protected] ~]# 

Now, we have installed our JDK environment.

For the other two servers, we don't need to configure them first. Later, we can use SCP directly.
Some friends may find that I named hadoop004 During the creation process. This is because I did a good job when I was doing hadoop000. I forgot to save it and lost it, this is my supplement. The code word is not easy. Do not spray it if you do not wash it.

Here, the configured centos cluster is:
Hadoop000 192.168.1.61
Hadoop001 192.168.1.62
Hadoop002 192.168.1.63
Next, we can start building a hadoop cluster. the following address is:
Http://blog.csdn.net/liuge36/article/details/78613910

Environment Building-centos cluster building