Yarn Load local library Throw unable to load Native-hadoop library how to Troubleshoot

Source: Internet
Author: User
Tags builtin file system svn linux

With the official Hadoop 2.1.0-beta installed, every time the Hadoop command goes in, it throws a warning

WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable

Set the logger level to see the specific reason

Export Hadoop_root_logger=debug,console

The 
 13/08/29 13:59:38 DEBUG util.  
Nativecodeloader:trying to load the custom-built Native-hadoop library ... 13/08/29 13:59:38 DEBUG util. nativecodeloader:failed to load Native-hadoop with Error:java.lang.UnsatisfiedLinkError:/usr/local/hadoop/ hadoop-2.1.0-beta/lib/native/libhadoop.so.1.0.0:/usr/local/hadoop/hadoop-2.1.0-beta/lib/native/ Libhadoop.so.1.0.0:wrong ELF class:elfclass32 (Possible cause:architecture Word width mismatch) 13/08/29 13:59:38 DEB UG util. nativecodeloader:java.library.path=/usr/local/hadoop/hadoop-2.1.0-beta/lib/native/linux-amd64-64:/usr/local/ Hadoop/hadoop-2.1.0-beta/lib/native 13/08/29 13:59:38 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable

Wrong ELFCLASS32, could it be that the loaded so file system version is wrong
Execute command File libhadoop.so.1.0.0
Libhadoop.so.1.0.0:elf 32-bit LSB Shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped
It's 80386, it's a 32-bit system version, and my Hadoop environment is 64-bit OS
Originally downloaded from the Apache image of the compiled Hadoop version of the native library are version 32, if you want to support the 64-bit version, you must recompile, this is a bit of a pit dad, you know, almost all production environment is 64-bit OS
Yarn's official statement to the native library validates this.
"The pre-built 32-bit i386-linux native Hadoop library is available as part of the Hadoop distribution and are located in T He lib/native directory "

More Wonderful content: http://www.bianceng.cnhttp://www.bianceng.cn/webkf/tools/

Re-checkout source code
SVN checkout http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta/
Plus the option to compile native, the compile-time will produce the corresponding native library based on the current operating system architecture
MVN Package-pdist,native-dskiptests-dtar
Then go to native file to see all the files type, is already 64-bit version of the, replace the online file, warning eliminate

File *  
libhadoop.a: current        ar archive  
libhadooppipes.a: current   ar archive  
libhadoop.so:       Symbolic link to ' libhadoop.so.1.0.0 '  
libhadoop.so.1.0.0:elf 64-bit LSB Shared Object, x86-64, version 1 (gnu/linux), Dynamically linked, not stripped  
libhadooputils.a: current   ar archive libhdfs.a  
: current          ar Archive  
libhdfs.so:         symbolic link to ' libhdfs.so.0.0.0 '  
libhdfs.so.0.0.0:   ELF 64-bit LSB shared Object, x86-64, version 1 (gnu/linux), dynamically linked, not stripped

Author: csdn Blog Lalaguozhe

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.