Configuring the Hadoop environment under Mac

Download Hadoop (i downloaded the 2.8.0 version)

Click here to download

After downloading, use

TAR-ZXVF Tar Package name

Unzip the tar package, after the decompression has been completed there is such a file appears

Modify the configuration file for Hadoop

Configuration files that need to be modified are in the Etc/hadoop under the Hadoop installation path

Files that need to be modified include

• hadoop-env.sh
• Core-site.xml
• Hdfs-site.xml
• Mapred-site.xml
• Yarn-site.xml

Hadoop-env.sh Modify the contents as follows

Export JAVA_HOME=/LIBRARY/JAVA/JAVAVIRTUALMACHINES/JDK1. 7. 0_79.jdk/contents/homeexport hadoop_conf_dir=/users/fengzhen/desktop/hadoop/hadoop-2.8 . 0/etc/hadoop

Locate the two lines in the corresponding location and modify the paths in the native.

The configuration of the Core-site.xml is modified as follows

<configuration><property><name>fs.defaultfs</name><value>hdfs://  localhost:9000</value></property><!--used to specify the directory in which the Hadoop runtime generates files to  create the--><property> itself <name>hadoop.tmp.dir</name><value>/users/fengzhen/desktop/hadoop/hadoop-2.8. 0/tmp</value></property></configuration>

The configuration of the Hdfs-site.xml is modified as follows

<configuration><property><!--Specifies the number of HDFs saved copies of data, including itself, by default 3--><!--pseudo-distributed mode, which must be 1--><name >dfs.replication</name><value>1</value></property><!-- Not root user can also write files to hdfs--><property><name>dfs.permissions</name><value>false </value>    <!--close firewall--></property></configuration>

The configuration of the Mapred-site.xml is modified as follows

<configuration><property><!--Specify mapreduce to run on yarn--><name>mapreduce.framework.name</ Name><value>yarn</value></property></configuration>

The configuration of the Yarn-site.xml is modified as follows

<configuration><!--Site Specific YARN configuration Properties--><property><name> Yarn.resourcemanager.hostname</name><value>localhost</value></property><property ><!--NodeManager How to get Data--><name>yarn.nodemanager.aux-services</name><value> Mapreduce_shuffle</value></property></configuration>

Configuring the Hadoop environment variables

Configure the Hadoop environment variable in the. bash_profile file and use Vim to open the file and enter edit mode

Vim ~/.bash_profile

In this file, add

Export hadoop_home=/users/fengzhen/desktop/hadoop/hadoop-2.8.0   Here is the installation path for HADOOP export path= $PATH: $HADOOP _ Home/bin

Save exit after adding complete.

Use the source command to make the changes in the file effective immediately

SOURCE ~/.bash_profile

All of this configuration is complete and the command is then formatted Namenode

Hadoop Namenode-format

After the format is successful, you can start Hadoop, there is a/sbin directory under the Hadoop installation directory, there is a start-all.sh in the directory, execute the file

./start-all.sh

You will need to enter the password several times during the startup process.

Output all Java processes and their PID at the Shell Input command JPS after the boot is complete

At this point, the Hadoop environment is configured successfully.

At this point, execute the following command

Hadoop fs-mkdir/testdata     Creates a folder under the HDFs root directory testdata
Hadoop fs-ls/can view files and folders under the root directory

End..

Configuring the Hadoop environment under Mac