1, Problem permission/tmp/hadoop ....Workaround: Download the Hadoop source package and modify the method Checkreturnvalue content in the Org.apache.hadoop.fs.FileUtil.java file. Change the exception to log. Intent: Not detecting file system path problemsprivate static void Checkreturnvalue (BOOLEANRV, File p, Fspermission permission ) throws IOException {if (!RV) {Log.debug ("Failed to set permissions of path:" + p+"To" +String.Format ("O", Permission.toshort ()));}}Replacement process: (A friend is also under Eclipse, and then replaced, but also held a mistake, because the jar package directly to the project's liberty, so I added the replacement process here.) )Under eclipse It's not easy to add to liberaries below. The Eclipse Project uses the Ivy plugin to manage the jar package, so the introduction of all third-party jars is the work of Ivy.Ivy has a default path to store jar files, can be modified, as for how to modify I am not here to elaborate, the default path is C:\Documents Andsettings\administrator\.ivy2\cache,So when we replace the jar of Hadoop, we also change the replacement here, there are two ways to replace it here:One is to open the jar directory directly under the Hadoop directory and replace the one version of the jar used by the project (see Ivy.xml under the project),and modify the name of the replaced JAR file to the name of the previous jar.The other is like the other jar files, which files are copied which files, and then change the file name to our recompiled jar filename, put our recompiled jar under the Jar folder.Note the path to the jar that needs to be extracted after recompiling Hadoop, and don't take the wrong. Must be under the build directory.
2, Org.apache.gora.sql.store.SqlStore notfound problemThis problem only exists in nutch2.2, the configuration file in 2.1 Ivy.xml has been configured to rely on, so this problem does not occur. This kind of problem is still com.healthmarketscience.sqlbuilder.dbspec.Column to be found. Workaround: Configure Ivy.xml to remove the comments from the following line.
4.
com.mysql.jdbc.DriverNotFound a problem;Cause: The attribute value in the Gora.properties file cannot have any spaces, otherwise there is an inexplicable problem. Nutch's fault-tolerant mechanism is less rigorous. Workaround: View the Gora.properties file property value, whether there is a problem with the configuration, especially the space. Strict troubleshooting. No front and rear spaces are OK. I've been doing it for 2 days because of this space.
5. Issue: Job Failed:name=inject urls,jobid=job_local_0001Just started to build, similar problems occur at the most frequency, similar to the issue of the first thought should be hadoop.log this file, rather than just look at the console. The specific problem is treated by hadoop.log the log record file. The following workaround is just one of the problems I have encountered in the expediency, for reference only. Don't mislead us. Summary: This type of problem typically occurs because the program catches an exception and then interrupts the job unexpectedly. As long as the specific problem is found in the Hadoop.log, there is generally no problem. Workaround: Two kinds. 1). Remove the configuration of the plugin in Nutch-site.xml. Do not use plug-ins without error, there may be some plug-in problems. This situation can be ruled out after Ant executes the nutch under Runtime/local/bin. 2). nutch2.2 to modify the version of the Gora-core dependency package in Ivy.xml, modify it to 0.2.1
6.java.lang.outofmemoryerror:java Heap SpaceThis problem is the direct cause of the 5th problem. The problem description is seen in the Hadoop.log. Workaround: Run the Parameters VM arguments add:-xms800m-xmx800m
7. View the problem method: Do not always look at the console, the information of the console does not completely solve the problem, and can not fully locate the problem, only through the root directory of the Hadoop.log file to view the log information.
8.
x Point Org.apache.nutch.net.URLNormalizer notfound. This mistake was encountered when a friend tested Nutch. Similar to the problem, see if the plugin is not a class, if the Nutch plug-in class, you can be sure that these errors are because the plug-in is not found the reason. Solution: After the build project, you must copy the build folder under the project's root directory to the root of the project. Then refresh the project in Eclipse. You will then need to modify a configuration parameter. Modify the property value of the Plugin.folders property in Nutch-site.xml. As follows:Plugin.folders./plugins
9.java.lang.nosuchmethoderror:org.apache.gora.persistency.persistent.getschema () Lorg/apache/avro/Schema
Solved. You must add correct version of Gora-hbase to you libraries. Gora-hbase-0.3.jar
Summary: The first contact, the console to the problem is not comprehensive, only debug, and later found to watch "Hadoop.log" file can clearly see the problem, this is very important, again mark!
nutch2.x Solutions to some of the problems encountered in running the eclipse+windows environment
