Configure Hadoop0.20.2 in UbuntuLinux

Configure ssh to create a key. Here, p is followed by an empty password. We do not recommend using an empty password ssh-keygen-trsa-p'' cat ~ /. Ssh/id_dsa.pub & gt ;~ //. Ssh/authorized_keys after configuration, run the sshlocalhost command to confirm that your machine can be connected using SSH, and you do not need to manually enter a password to download Hadoopwgethttp: // mirror.

Configure ssh

Create a key. Here, p is followed by an empty password. It is not recommended to use a blank password.

Ssh-keygen-t rsa-p''

Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys

After the configuration is complete, execute ssh localhost to confirm that your machine can be connected using SSH, and you do not need to manually enter the password during connection.

Download Hadoop

Wget http://mirror.bjtu.edu.cn/apache/hadoop/common/hadoop-0.20.2/hadoop-0.20.2.tar.gz

Tar-xvf hadoop-0.20.2.tar.gz

Add environment variable

In/etc/environment

HADOOP_HOME = directory path

JAVA_HOME = jdk path

Add/etc/profile

Export HADOOP_HOME = directory path

Export JAVA_HOME = jdk path

Modify the hadoop configuration file

Modify $ HADOOP_HOME/conf/hadoop-env.sh

# Export JAVA_HOME =

Change to export JAVA_HOME = jdk path

Set HDFS directory

Mkdir/$ HOME/tmp

Chmod-R 777/$ HOME/tmp

Modify the/conf/core-site.xml File

Add

Hadoop. tmp. dir
/Home/du/tmp
A base for other temporary directories.


Fs. default. name
Hdfs: // localhost: 54310

Change conf/mapred-site.xml

Mapred. job. tracker
Localhost: 54311
The host and port that the MapReduce job tracker runs
At. If "local", then jobs are run in-process as a single map
And reduce task.

Change conf/hdfs-site.xml

Add

Dfs. replication
1
Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.

Format a File System

Hadoop namenode-format

Start pseudo-distributed

$ HADOOP_HOME/bin/start-all.sh

Check whether startup is successful

Jps

18160 SecondaryNameNode
17777 NameNode
17970 DataNode
Jps 18477
18409 TaskTracker
18231 JobTracker

Stop

$ HADOOP_HOME/bin/stop-all.sh

Hadoop web access interface

Http: /localhost: 50030/job tracker access
Http: // localhost: 50060/task tracker access
Access http: // localhost: 50070/name node