Hadoop1.2.1 Installation notes 3: hadoop Configuration

• Create a hadoop folder under the/usr directory and grant the hadoop user permission (master)

[[email protected] usr]$ sudo mkdir hadoop[[email protected] usr]$ ls -al total 156 drwxr-xr-x. 2 root root 4096 Jul 31 00:17 hadoop [[email protected] usr]$ sudo chown -R hadoop:hadoop hadoop[[email protected] usr]$ ls -al total 156 drwxr-xr-x. 2 hadoop hadoop 4096 Jul 31 00:17 hadoop

• Install hadoop in the/usr/hadoop directory, and then delete the installation package

[[email protected] hadoop]$ tar -zxvf hadoop-1.2.1.tar.gz[[email protected] hadoop]$ ls hadoop-1.2.1 hadoop-1.2.1.tar.gz [[email protected] hadoop]$ rm hadoop-1.2.1.tar.gz [[email protected] hadoop]$ ls hadoop-1.2.1

• Configure the hadoop variable/etc/profile

#hadoop environmentexport HADOOP_HOME=/usr/hadoop/hadoop-1.2.1export PATH=$PATH:$HADOOP_HOME/bin"/etc/profile" 85L, 2070C written

• Create the TMP folder in the hadoop Directory, which needs to be set in the core-site.xml, otherwise it will be deleted after restart, you must format the node again

[[email protected] hadoop-1.2.1]$ mkdir tmp [[email protected] hadoop-1.2.1]$ ls -al total 8404 drwxrwxr-x. 2 hadoop hadoop 4096 Jul 31 00:44 tmp

• Configure the Java variable and specify the Java directory

[[email protected] conf]$ pwd /usr/hadoop/hadoop-1.2.1/conf [[email protected] conf]$ vim hadoop-env.sh#JAVA environment setting export JAVA_HOME=/usr/java/jdk1.7.0_65

The hadoop configuration file is in the/usr/hadoop/hadoop-1.2.1/conf directory,

Code development is divided into core, HDFS and MAP/reduce three parts, core-site.xml, hdfs-site.xml, mapred-site.xml.

Core-site.xml and hdfs-site.xml are profiles from the HDFS perspective; core-site.xml and mapred-site.xml are profiles from the mapreduce perspective.

• Configure the core-site.xml, set the HDFS access path and port, and set the temporary file directory

[[email protected] conf]$ vim core-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <!-- file system properties --> <property> <name>fs.default.name</name> <value>hdfs://10.15.5.200:9000</value> </property> <property>        <name>hadoop.tmp.dir</name>        <value>/usr/hadoop/hadoop-1.2.1/tmp</value>         <description>temporary directories.</description>    </property></configuration>

• Configure the hdfs-site.xml to redefine the number of replication replicas. By default, 3 slaves is less than 3, and the system reports an error

[[email protected] conf]$ vim hdfs-site.xml <value>1</value> <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> </configuration>

• Configure mapred-site.xml

[[email protected] conf]$ vim mapred-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>mapred.job.tracker</name> <value>10.15.5.200:9001</value> </property> </configuration>

Configure master and slaves.

[[email protected] conf]$ vi masters 10.15.5.200[[email protected] conf]$ vi slaves 10.15.5.201 10.15.5.202

--------------- Install hadoop on slaves and configure the environment ----------------------------------------------------

When using the root account to create/usr/hadoop with hadoop user permissions, you can switch back to hadoop to prevent other unnecessary problems.