Hadoop installation and configuration

Hadoop pseudo-distributed Installation Steps

Log on as a root user

1.1 set static IP

Right-click the icon in the upper-right corner of the centos desktop and choose modify.

Restart the NIC and run the Command Service Network restart.

Verify: Execute the command ifconfig

 

1.2 Modify host name

<1> modify the host name in the current session and run the command hostname hadoop (the host name to be modified)

<2> modify the host name in the configuration file and run the command VI/etc/sysconfig/network.

Verification: restart the machine

1.3 bind the hostname and IP address

Run the command VI/etc/hosts and add a line as follows:

192.168.1.112 hadoop

Save and exit

 

Verification: Ping hadoop

 

 

1.4 disable Firewall

Run service iptables stop

Verify: Service iptables status

 

1.5 disable automatic firewall execution

Run the command chkconfig iptables off.

Verification: chkconfig -- list | grep iptables

 

1.6ssh (Secure Shell)

 

Execute the command ssh-keygen-t rsa to generate the key, which is located ~. /Ssh folder

 

Execute Command CP ~ /. Ssh/id_rsa.pub authorized_keys

Verify SSH localhost

1.7 install JDK

Use winscp to copy JDK files from windows to the/usr/local directory

Run the command chmod U + x JDK to grant the execution permission

Run the command./JDK. bin to decompress the package.

Run the command mv jdk formerly JDK rename

Run VI/etc/profile to set the environment variable and add two lines.

Export ava_home = JDK installation directory

Export Path =.: $ java_home/bin: $ path

Save and exit

Run the source/etc/profile command to make the settings take effect immediately.

Verification: Java-version

 

1.8 install hadoop

Run the tar-zxvf hadoop-1.1.2.tar.gz to extract

Execute Command MV hadoop-1.1.2 hadoop rename

Set Environment Variables

Add a line to the environment variable in VI/etc/profile settings

Export hadoop_home = hadoop installation directory

Modify a line of content

Export Path =.: $ java_home/bin: $ hadoop_home: $ path

Save and exit

Run the source/etc/profile command to make the settings take effect immediately.

 

Modify the hadoop configuration file in the $ hadoop_home/conf folder.

Modify four configuration files:

Hadoop-env.sh

Core-site.xml

Hdfs-site.xml

Mapred-site.xml

Run the hadoop namenode-format command to format hadoop.

Run the command start-all.sh to start

Verification: 1. Execute the command JPs and find five Java processes: namenode, datanode, secondaryname, jobtracker, tasktracker

2. view in a browser

Hadoop: 50070