Hadoop Detailed Configuration Tutorial

Use putty or Xshell to connect to a remote host under Windows

Mac uses terminal to connect to remote Linux host:ssh [email protected]

User is the administrator name for the Linux server

Hostname is the IP of the Linux server

Attention:

PWD shows where the current directory is located.

Use the WHO command to view the currently logged on user, and use the W command to see all logged-in user information.

View host name hostname command: hostname or UNAME-N

View system version, kernel, and more: uname-a

How Linux looks at the number of CPUs and total memory of the host:Cat/proc/cpuinfo and Cat/proc/meminfo

install JDK
# ls
# Javac//Detect Java Environment (prompted, install as prompted)
# sudo Apt-get install OPENJDK-7-JDK
# Javac//Detect Java Environment
# vim/etc/profile
Keyboard Input A, enter edit mode, edit file, set Java environment variable
Export java_home=/usr/lib/jvm/java-7-openjdk-amd64/
Export jre_home= $JAVA _home/jre
Export classpath= $JAVA _home/ Lib: $JRE _home/lib: $CLASSPATH
Export path= $JAVA _home/bin: $JRE _home/bin: $PATH
Point ESC exit Edit mode
Input: Wq Save exit
# Source/etc/profile (make it Effective)
# javac//Detecting the Java environment

Installing Hadoop
Download Hadoop:
# wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz
# ls
# MV hadoop-1.2.1.tar.gz/opt/(Put the Hadoop installation package in the OPT directory, note: mkdir is the Create Folder command)
# cd/opt/
# ls
# TAR-ZXVF hadoop-1.2.1.tar.gz (uncompressed)
# ls
# CD HADOOP-1.2.1/
# ls
(The main configuration folder is conf)
# CD conf/
# ls
Main Configuration 4 files: Mapred-site.xml, Core-site.xml, Hdfs-site.xml, hadoop-env.sh
# Vim Hadoop-env.sh
Edit has export=java_home that line: Remove the front #, export JAVA_HOME=/USR/LIB/JVM/JAVA-7-OPENJDK-AMD64
If you have forgotten the address of the JAVA environment variable, exit and then use the # echo $JAVA _home to see the variable address and copy it down to paste. Last: Wq Save

# Vim Core-site.xml (Note: IMOOC is its own native hostname that is the hostname, 9000 is the port number)
Configuration content:
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/hadoop</value>
</property>

<property>
<name>dfs.name.dir</name>
<value>/hadoop/name</value>
</property>

<property>
<name>fs.default.name</name>
<value>hdfs://imooc:9000</value>
</property>

</configuration>

: Wq Save Exit

# Vim Hdfs-site.xml
Configuration content:
<configuration>
<property>
<name>dfs.data.dir</name>
<value>/hadoop/data</value>
</property>

</configuration>

: Wq Save Exit

# Vim Mapred-site.xml
Configuration content:
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>imooc:9001</value>
</property>

</configuration>

: Wq Save Exit

# vim/etc/profile (Hadoop environment variable configuration)
Export hadoop_home=/opt/hadoop-1.2.1
Export path= $JAVA _home/bin: $JRE _home/bin: $HADOOP _home/bin: $PATH
: Wq Save Exit

# Source/etc/profile (setting takes effect)
# Hadoop (Detect Hadoop)
# CD:
# CD bin/(i.e./opt/hadoop-1.2.1/bin directory)
# ls
# Hadoop Namenode-format (format operation for Namenode)

Start Hadoop:
# cd/opt/hadoop-1.2.1/bin/
# ls
# start-all.sh
# JPS (use JPS to see if Hadoop is working and which processes are running)

# Hadoop Fs-ls/(see what files are under Hadoop)

Hadoop Detailed Configuration Tutorial