In linux, from JDK installation to ssh installation to hadoop standalone pseudo distributed deployment

Environment: ubuntu10.10JDK1.6.0.27hadoop0.20.2 I. JDK installation in ubuntu: 1. download jdk-6u27-linux-i586.bin2. copy to/usr/java and set the object operation permissions. $. /jdk-6u27-linux-i586.bin start installation 4. set...

 

Environment: ubuntu 10.10 JDK1.6.0.27 hadoop 0.20.2

 

1. install JDK in ubuntu:

 

1. download jdk-6u27-linux-i586.bin

 

2. copy to/usr/java and set the object operation permission.

 

3. $./jdk-6u27-linux-i586.bin start installation

 

4. set the environment variable vi/etc/profile to add at the end of the file

 

JAVA_HOME =/usr/Java/jdk1.6.0 _ 27

PATH = $ JAVA_HOME/bin: $ PATH

CLASSPATH =.: $ JAVA_HOME/lib/tools. jar: $ JAVA_HOME/lib/dt. jar

Export PATH JAVA_HOME CLASSPATH

5. set the JDK installed by the user to the default JDK and run

 

$ Update-alternatives -- install/usr/bin/java/usr/lib/jvm/java/jdk1.6.0 _ 12/bin/java 300

$ Update-alternatives -- install/usr/bin/javac/usr/lib/jvm/java/jdk1.6.0 _ 12/bin/javac 300

 

$ Update-alternatives -- config java

 

6. input java-version to install the SDK.

 

II. install ssh in ubuntu:

 

1. $ sudo apt-get install openssh-server

2. start ssh/etc/init. d/ssh start

3. $ ps-e | grep ssh to verify whether sshserver is enabled

 

4. password-free

 

$ Ssh-keygen-t dsa-p'-f ~ /. Ssh/id_dsa

$ Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys

 

3. install hadoop in ubuntu:

 

1. download hadoop-0.4102.tar.gz, put in/usr/hadoop http://apache.etoak.com//hadoop/core/

 

2. decompress $ tar zxvf hadoop-0.20.2.tar.gz

 

3. modify the hadoop configuration file

 

Conf/hadoop-env.sh modify JAVA_HOME options:

 

Export JAVA_HOME =/usr/java/jdk.1.6.0 _ 27

 

4. pseudo-distributed single-host configuration

 

Conf/core-site.xml

 

 

 

 

 

Fs. default. name  

Hdfs: // localhost: 9000  

 

 

 

Conf/hdfs-site.xml:

 

 

 

 

Dfs. replication  

1  

 

 

 

 

Conf/mapred-site.xml:

 

 

 

 

 

Mapred. job. tracker  

Localhost: 9001  

 

 

 

5. create an hdfs file system

$ Bin/hadoop namenode-format

 

6. start ssh $/etc/init. d/ssh start before starting hadoop.

$ Bin/hadoop start-all.sh

 

7. create a test directory under hadoop, and then create file1.txt file2.txt to write a few words. Upload The test files under hadoop/test to the hadoop file system.

 

$ Bin/hadoop dfs-put./test input

 

8. example of running wordCount

 

$ Bin/hadoop jars hadoop-0.20.2-examples.jar wordcount input output

 

9. copy the result from dfs

 

$ Bin/hadoop dfs-get output

 

10. View Results

 

$ Cat output/* You can also directly view $ bin/hadoop dfs-cat output /*

 

11. stop hadoop running

 

$ Bin/hadoop stop-all.sh

 

12. disable ssh-server

 

$/Etc/init. d/ssh stop

From: daniel's column