hadoop2.4 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable

Source: Internet
Author: User
Tags builtin hadoop fs

After installing hadoop2.4 on Ubuntu, use the following command:

 Hadoop fs-ls  14 /09 /09  11 : 33 : 51  WARN util. nativecodeloader:unable to load Native-hadoop library for  your platform ... using Builtin-java classes where Applicablefound  1   itemsdrwxr -xr-x-duomai supergroup 0  09 -05  12 : 10  Flume 

There is a warn hint that Hadoop can not load the local library, the Internet search, the problem is basically due to the Apache Hadoop official online download hadoopXXX.bin.tar.gz really 32-bit machine compiled (egg pain bar), I use the cluster 64bit, Error loading. So files, of course, does not affect the use of Hadoop (if you use Mahout to do some machine learning tasks may encounter trouble, loading is not successful, the task exits directly, so it is necessary to solve the warn).

But every time you run a command, there's a lot of warn, and it's easy to kill it:

1 . Download hadoop2.4 Source code 2 . Compiling on a machine on a cluster 3. Replace the previous $hadoop_home/lib/native with the newly compiled native

Here are a few things to note:

(1) before 2 (compile), first modify the MAVEN installation directory Conf/settings.xml (because MAVEN used foreign reposity, domestic sometimes inaccessible, modified as a domestic mirror), <mirrors></ Mirros> Add, the other does not need to change, the specific changes are as follows:

   <Mirror>        <ID>Nexus-osc</ID>         <mirrorof>*</mirrorof>     <name>Nexusosc</name>     <URL>http://maven.oschina.net/content/groups/public/</URL>   </Mirror>

Similarly, new additions within <profiles></profiles>

< Profile>       <ID>jdk-1.7</ID>       <activation>         <JDK>1.7</JDK>       </activation>       <repositories>         <Repository>           <ID>Nexus</ID>           <name>Local private Nexus</name>           <URL>http://maven.oschina.net/content/groups/public/</URL>           <releases>             <enabled>True</enabled>           </releases>           <Snapshots>             <enabled>False</enabled>           </Snapshots>         </Repository>       </repositories>       <pluginrepositories>         <pluginrepository>           <ID>Nexus</ID>          <name>Local private Nexus</name>           <URL>http://maven.oschina.net/content/groups/public/</URL>           <releases>             <enabled>True</enabled>           </releases>           <Snapshots>             <enabled>False</enabled>           </Snapshots>         </pluginrepository>       </pluginrepositories>     </ Profile>

Compile Hadoop with the following command after the modification is complete;

MVN package-dmaven.javadoc.skip=true -pdist,native-dskiptests-dtar

And then just wait, about 20min later, build success, Target in

Hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1.tar.gz

(2) After the successful compilation, the new lib/native replaced the original lib/native in the cluster, remember to modify the $hadoop_home/etc/hadoop/hadoop-env.sh, at the end of the addition;

Export hadoop_opts="-djava.library.path= $HADOOP _home/lib: $HADOOP _home/lib/native"

======

Run the following command again:

Hadoop FS-ls1  itemsdrwxr-xr-x   -duomai supergroup          02014 -flume:Ten

Warn disappears, good!

Resources:

1. http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-common/NativeLibraries.html

2. http://stackoverflow.com/questions/19943766/ Hadoop-unable-to-load-native-hadoop-library-for-your-platform-error-on-centos

3. http://blog.csdn.net/w13770269691/article/details/16883663/#

hadoop2.4 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.