CentOS 5.10 installation hadoop-1.2.1

CentOS 5.10 installation hadoop-1.2.1
System Environment: CentOS 5.10 (Virtual Machine) [plain] view plaincopyprint?

1. [Root @ localhosthadoop] # lsb_release-a
2. LSBVersion: core-4.0-ia32: core-4.0-noarch: graphics-4.0-ia32: graphics-4.0-noarch: printing-4.0-ia32: printing-4.0-noarch:
3. DistributorID: CentOS
4. Description: CentOSrelease5.10 (Final)
5. Release: 5.10
6. Codename: Final

Prepare Jdk installation and configuration Oracle official website download jdk, here I download is the jdk-6u45-linux-i586.bin, and upload to the virtual machine, use the root user, execute the following command to create a folder, move the installation file, execute the installation. [Plain] view plaincopyprint?

1. Mkdir/usr/java
2. Music Video/home/auxu/Desktop/jdk-6u45-linux-i586.bin/usr/java
3. Cd/usr/java
4. /Jdk-6u45-linux-i586.bin

Configure the environment variable [plain] view plaincopyprint?

1. Vi/etc/profile

Add export JAVA_HOME =/usr/java/jdk1.6.0 _ 45
Export JRE_HOME = $ JAVA_HOME/jre
Export CLASSPATH =.: $ JAVA_HOME/lib/dt. jar: $ JAVA_HOME/lib/tools. jar
Export PATH = $ PATH: $ JAVA_HOME/bin after saving, run: [SQL] view plaincopyprint?

1. Source/etc/profile

Verify java configuration [SQL] view plaincopyprint?

1. [Root @ localhostjava] # java-version
2. Javaversion "1.6.0 _ 45"
3. Java (TM) SERuntimeEnvironment (build1.6.0 _ 45-b06)
4. JavaHotSpot (TM) ClientVM (build000045-b01, mixedmode, sharing)

You can simply write a java class for testing.

Create a Hadoop user and Related Application folders. Use the root user to create a new user named hadoop [plain] view plaincopyprint?

1. Useraddhadoop
2. Passwdhadoop

Create an application folder for later hadoop configuration [SQL] view plaincopyprint?

1. Mkdir/hadoop
2. Mkdir/hadoop/hdfs
3. Mkdir/hadoop/hdfs/data
4. Mkdir/hadoop/hdfs/name
5. Mkdir/hadoop/mapred
6. Mkdir/hadoop/mapred/local
7. Mkdir/hadoop/mapred/system
8. Mkdir/hadoop/tmp

Change the folder owner to hadoop user [plain] view plaincopyprint?

1. Chown-Rhadoop/hadoop

Set a Hadoop user to enable ssh to localhost [SQL] view plaincopyprint without a password?

1. Su-hadoop
2. Ssh-keygen-tdsa-p'-f ~ /. Ssh/id_dsa
3. Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys
5. Cd/home/hadoop/. ssh
6. Chmod600authorized_keys

Note the permission issues here. Ensure that the. ssh directory permission is 700, and the authorized_keys is 600 for verification: [plain] view plaincopyprint?

1. [Hadoop@localhost.ssh] $ sshlocalhost
2. Lastlogin: Sun nov1722: 11: 552013

Ssh localhost can be connected without a password. Configure OK
Install and configure Hadoop, create a directory, install and switch back to the root user, and create the installation directory [plain] view plaincopyprint?

1. Su

[Plain] view plaincopyprint?

1. Mkdir/opt/hadoop

Move the installation file to the new directory above to ensure its execution permission, and then run [plain] view plaincopyprint?

1. Music Video/home/auxu/Desktop/hadoop-1.2.1.tar.gz/opt/hadoop
2. Cd/opt/hadoop
3. Tar-xzvfhadoop-1.2.1.tar.gz

Change the owner of the hadoop installation directory to [plain] view plaincopyprint?

1. Chown-Rhadoop/opt/hadoop

Switch to the hadoop user and modify the configuration file. Here, the configuration is based on the previously created application file, and the [plain] view plaincopyprint?

1. Su-hadoop

[Plain] view plaincopyprint?

1. Cd/opt/hadoop/hadoop-1.2.1/conf

Core-site.xml [html] view plaincopyprint?

1. <Configuration>
2. <Property>
3. <Name> fs. default. name </name>
4. <Value> hdfs :/// localhost: 9000 </value>
5. </Property>
6. <Property>
7. <Name> hadoop. tmp. dir </name>
8. <Value>/hadoop/tmp </value>
9. </Property>
10. </Configuration>

Hdfs-site.xml [html] view plaincopyprint?

1. <Configuration>
2. <Property>
3. <Name> dfs. replication </name>
4. <Value> 1 </value>
5. </Property>
6. <Property>
7. <Name> dfs. name. dir </name>
8. <Value>/hadoop/hdfs/name </value>
9. </Property>
10. <Property>
11. <Name> dfs. data. dir </name>
12. <Value>/hadoop/hdfs/data </value>
13. </Property>
14. </Configuration>

Mapred-site.xml [html] view plaincopyprint?

1. <Configuration>
2. <Property>
3. <Name> mapred. job. tracker </name>
4. <Value> localhost: 9001 </value>
5. </Property>
6. </Configuration>

The hadoop-env.sh configures JAVA_HOME and HADOOP_HOME_WARN_SUPPRESS. PS: HADOOP_HOME_WARN_SUPPRESS this variable can avoid such a reminder in some situations: "WARM: HADOOP_HOME is deprecated" [plain] view plaincopyprint?

1. ExportJAVA_HOME =/usr/java/jdk1.6.0 _ 45
2. ExportHADOOP_HOME_WARN_SUPPRESS = "TRUE"
3. Sourcehadoop-env.sh

Reconfigure the/etc/profile file, for example, [html] view plaincopyprint?

1. ExportJAVA_HOME =/usr/java/jdk1.6.0 _ 45
2. ExportJRE_HOME = $ JAVA_HOME/jre
3. ExportCLASSPATH =.: $ JAVA_HOME/lib/dt. jar: $ JAVA_HOME/lib/tools. jar
4. ExportHADOOP_HOME =/opt/hadoop/hadoop-1.2.1.
5. ExportPATH = $ PATH: $ JAVA_HOME/bin: $ HADOOP_HOME/sbin

Apply the updated configuration file to [html] view plaincopyprint?

1. Source/etc/profile

Test hadoop's installation of [html] view plaincopyprint?

1. [Hadoop @ localhostconf] $ hadoopversion
2. Hadoop1.2.1
3. Subversionhttps: // svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2-r1503152
4. CompiledbymattfonMonJul2215: 23: 09PDT2013
5. Fromsourcewithchecksum6923c86528809c4e7e6f493b6b413a9a

To start HADOOP, You Need To Format namenode first, and then start all services [html] view plaincopyprint?

1. Hadoopnamenode-format
2. Start-all.sh

View the process [html] view plaincopyprint?

1. Hadoop @ localhostconf] $ jps
2. 6360 NameNode
3. 6481 DataNode
4. 6956Jps
5. 6818 TaskTracker
6. 6610 SecondaryNameNode
7. 6698 JobTracker

If you can find these services, Hadoop has been successfully started. If there is any problem, you can go to/opt/hadoop/hadoop-1.2.1/logs to view the corresponding log, and finally you can access the haddop service through the following link.
Localhost: 50030/for the Jobtracker
Localhost: 50070/for the Namenode
Localhost: 50060/for the TasktrackerHadoop JobTracker: Hadoop Namenode: Hadoop TaskTracker: PS: Fully Distributed installation is similar to pseudo distributed installation. Pay attention to the following points: 1. ssh user-free logon in the cluster 2. specify the specific IP address (or machine name) in the configuration file, instead of configuring the masters and slaves files in localhost3. add the relevant IP address (or machine name) to ensure that the above configurations must be consistent on each node.