Introduction to Hadoop deployment under Mac (MacOSX10.8.3 + Hadoop-1.0.4)

OneCoder deploys the Hadoop environment on its own notebook for research and learning, recording the deployment process and problems encountered. 1. Install JDK. 2. Download Hadoop (1.0.4) and configure the JAVA_HOME environment variable in Hadoop. Modify the hadoop-env.sh file. ExportJAVA_HOMELibraryJavaJavaVirtualMachinesjdk1.7.0_10.jdk

OneCoder deploys the Hadoop environment on its own notebook for research and learning, recording the deployment process and problems encountered. 1. Install JDK. 2. Download Hadoop (1.0.4) and configure the JAVA_HOME environment variable in Hadoop. Modify the hadoop-env.sh file. Export JAVA_HOME =/Library/Java/JavaVirtualMachines/jdk1.7.0 _ 10.jdk/

OneCoder deploys the Hadoop environment on its own notebook for research and learning, recording the deployment process and problems encountered.

1. Install JDK.
2. Download Hadoop (1.0.4) and configure the JAVA_HOME environment variable in Hadoop. Modify the hadoop-env.sh file.
Export JAVA_HOME =/Library/Java/JavaVirtualMachines/jdk1.7.0 _ 10.jdk/ Contents/Home/
3. Configure SSH
Generate key

ssh-keygen -t dsa -P '' -f ~/.ssh/onecoder_dsa

Append the public key to the key.

cat ~/.ssh/onecoder_rsa.pub >> ~/.ssh/authorized_keys

Enable remote access for Mac OS. System settings-share-remote Logon

4. Configure the local paths of namenode and datanode hdfs
Configure in hdfs-site.xml

  
  
   dfs.name.dir
   
  
   /Users/apple/Documents/hadoop/name/
  
 
    
  
   dfs.data.dir
    
  
   /Users/apple/Documents/hadoop/data/
  
  
      
  
   dfs.replication
       
  
   1
    
 

Because it is a verification environment, data backup is set to 1.

5. Format namenode

bin/Hadoop NameNode -format

6. Start hadoop

Can be started directly through the start-all.sh under the bin, you can also through

hadoop namenode (datanode、jobtracker、tasktracker)

In the preceding order.
The latter allows you to conveniently view the startup log for troubleshooting. You can also view the access address and port on the monitoring page in the console log. For example:
13/04/04 15:52:18 INFO http. HttpServer: Jetty bound to port 50070
Of course, you can be familiar with these addresses. Go to the web Monitoring page in a browser. Everything is ready. Hadoop environment deployment is complete.

Others.
Warning: Unable to load realm info from SCDynamicStore
The solution offered on the Web is: set it in hadoop-env.sh

Export HADOOP_OPTS = "-Djava. security. krb5.realm = OX. AC. UK-Djava. security. krb5.kdc = kdc0.ox. ac. uk: kdc1.ox. ac. uk"

However, it is invalid in OneCoder, but does not affect the Hadoop environment.

Introduction to Hadoop deployment under Mac (Mac OSX 10.8.3 + Hadoop-1.0.4), thanks to the original author for sharing.