Install Hadoop1.0.0 in Ubuntu11.10 (standalone pseudo-distributed)

Environment configuration: Ubuntu11.10, Hadoop1.0.0 installation ssh1apt-getinstallssh installation rsync1apt-getinstallrsync configuration ssh password-free login 1ssh-keygen-tdsa-P & amp; #39; & amp; #39;-f ~ /. Ssh/id_dsa2cat ~ /. Ssh/id_dsa.pub

Environment configuration: Ubuntu11.10, Hadoop1.0.0

Install ssh

 
1
Apt-get install ssh
 
 
Install rsync
 
 
 
 
1
Apt-get install rsync
 
 
Configure ssh password-free login
 
 
 
 
1
Ssh-keygen-t dsa-p'-f ~ /. Ssh/id_dsa
2
Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys
 
 
Verify whether it is successful
 
 
 
 
1
Ssh localhost
 
 
 
 
Download Hadoop1.0.0, JDK
 
Create a linux terminal and an app directory. both Java and Hadoop are installed in this directory.
 
 
 
1
Mkdir/home/app
 
 
Next, install Java and Hadoop and decompress Hadoop.
 
 
 
 
1
Cd/home/app
2
Chmod + x jdk-6u30-linux-i586.bin
3
/Jdk-6u30-linux-i586.bin
4
 
5
Tar zxf hadoop-1.0.0-bin.tar.gz
 
 
Configure JDK environment variables
 
 
 
1
Vi/etc/profile
 
 
Add the following statement to the end
 
 
 
 
1
Export JAVA_HOME =/home/app/jdk1.6.0 _ 30
2
Export PATH = $ JAVA_HOME/bin: $ PATH
3
Export CLASSPATH =.: $ JAVA_HOME/lib/dt. jar: $ JAVA_HOME/lib/tools. jar
 
 
Configure Hadoop
 
 
Go to the Hadoop Directory
 
 
 
1
Cd/home/app/hadoop-1.0.0
 
 
Modify the configuration file and specify the JDk installation path.
 
 
 
 
1
Vi conf/hadoop-env.sh
 
 
1
Export JAVA_HOME =/home/app/jdk1.6.0 _ 30
 
 
Modify the Hadoop Core configuration file core-site.xml, which configures the address and port number of HDFS
 
 
 
 
1
Vi conf/core-site.xml
 
 
1

2

3
Fs. default. name
4
Hdfs: // localhost: 9000
5

6

 
 
Modify the HDFS configuration in Hadoop. the default backup mode is 3. because the single-host version is installed, you need to change it to 1.
 
 
 
 
1
Vi conf/hdfs-site.xml
 
 
1

2

3
Dfs. replication
4
1
5

6

 
 
Modify the MapReduce configuration file in Hadoop, which configures the address and port of JobTracker.
 
 
 
 
1
Vi conf/mapred-site.xml
 
 
1

2

3
Mapred. job. tracker
4
Localhost: 9001
5

6

 
 
Next, start Hadoop. before starting Hadoop, format the Hadoop file system HDFS, enter the Hadoop folder, and enter the following command
 
 
 
 
1
Bin/hadoop namenode-format
 
 
Start Hadoop and enter the command
 
 
 
 
1
Bin/start-all.sh
 
 
This command starts all services.
 
 
Finally, verify that Hadoop is successfully installed. Open your browser and enter the URL:
 
Http: // localhost: 50030 (MapReduce Web page)
 
Http: // lcoalhost: 50070 (HDfS web page)
 
If all data can be viewed, the installation is successful.

 

Hadoop divides hosts into two roles from three perspectives:

First, it is divided into master and slave, that is, master and slave.

Second, from the HDFS perspective, the host is divided into namenode and datanode (in distributed file systems, directory management is very important, and directory management is equivalent to the master, namenode is the directory manager ).

Third, from the perspective of MapReduce, the host is divided into JobTracker and TaskTracker (a job is often divided into multiple tasks, from which it is not difficult to understand the relationship between them ).