hadoop-(3) Hadoop issues Summary

1. The virtual machine installation hadoop,windows cannot access the Hadoop Web page http://master:50070/through the host name. Windows Ping Master also pings the method: Add Linux under Windows native C:\Windows\System32\drivers\etc\hosts files Hosts configure the hostname and IP address of the Hadoop machine to add in.

Issue 2, Windows Eclipse running Hadoop program, reported users inconsistent, no permissions org.apache.hadoop.security.AccessControlException: Org.apache.hadoop.security. Accesscontrolexception:permission Denied:user=sunzhaohua, Access=write, inode= "Szh": Method://1, this method of obtaining FS can specify the client that accesses HDFs as FS = Filesystem.get (New URI ("Hdfs:master:9000/"), conf, "Szh"); 2, reference: Http://www.huqiwen.com/2013/07/18/hdfs-permission-denied/windows in the system environment variable inside add Hadoop_user_name=szh, Restart Eclipse, then.

Issue 3, running the Hadoop fs-put command no ROOT to host method: found that the firewall is not shut down from the node, shut down the firewall can service iptables stop

Issue 4, run the./stop-all.sh command, JPS discovery process still exists, error no tasktracker to stop, no datanode to stop. Method: 1) kill-9 these processes to kill NETSTAT-NLTP 2) The problem is that Hadoop is based on the mapred and DFS process numbers on Datanode at stop. While the default process number is saved in/TMP, Linux defaults to deleting files from this directory at regular intervals (typically one months or 7 days). Therefore, after the deletion of Hadoop-hadoop-jobtracker.pid and hadoop-hadoop-namenode.pid two files, Namenode naturally can not find the two processes on the datanode. In the configuration file $hadoop_home/conf/hadoop-env.sh export hadoop_pid_dir=${hadoop_home}/pids can solve this problem

Problem 5, hadoop2.6.3 write MapReduce program, hit jar package run, Error Java.io.IOException:Type mismatch method: 1) kill-9 kill these processes can also NETSTAT-NLTP 2)//W         Ordcount.setmapoutputkeyclass (Text.class); Wordcount.setmapoutputkeyclass (Longwritable.class); The output type of map can be commented out

Problem 6, Local mode run Mapreduce,eclipse error: error [main] util. Shell (Shell.java:getWinUtilsPath (373))-Failed to locate the winutils binary in the Hadoop binary path Java.io.IOExcepti On:could not locate executable null\bin\winutils.exe in the Hadoop binaries. Method:   code settings in the Javamain method:   System.setproperty ("Hadoop.home.dir",  "e:/study/hadoop/hadoop-2.6.3" );

Issue 7, Local mode run Mapreduce,eclipse error: Exceptionin thread "main" Java.lang.unsatisfiedlinkerror:o Rg.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray (ii[bi[biiljava/lang/string; JZ) V This is due to the Hadoop.dll version issue, 2.4 before and after the need for a different Need to select the correct version and replace it on Hadoop/bin and C:\windows\system32 Download the hadoop2.6 plug-in package for the WINDOWS64-bit platform (Hadoop.dll,winutils.exe) In the hadoop2.6.0 source Hadoop-common-project\hadoop-common\src\main\winutils, there is a vs.net project, compiling this project can get this pile of files, output files, Hadoop.dll, Winutils.exe These two most useful, will winutils.exe copy to Hadoop\bin directory, Hadoop.dll copy to C:\Windows\System32 directory (mainly to prevent the plug-in reported various inexplicable errors, such as an empty object referencing what) Note: If you do not want to compile, you can download the compiled file directly hadoop2.6 (x64) V0.2.zip