Hadoop "Unable to load Native-hadoop library for Y

Http://devsolvd.com/questions/hadoop-unable-to-load-native-hadoop-library-for-your-platform-error-on-centos

The answer depends ... I just installed Hadoop 2.6 from Tarball on 64-bit CentOS 6.6. The Hadoop install did indeed come with a prebuilt 64-bit native library. For my install, it's here:

/opt/hadoop/lib/native/libhadoop.so.1.0.0

And I know it is 64-bit:

[Email protected] native]$ ldd libhadoop.so.1.0.0./libhadoop.so.1.0.0:/lib64/libc.so.6:version ' glibc_2.14 ' not Found (required by./libhadoop.so.1.0.0) Linux-vdso.so.1 = (0x00007fff43510000) libdl.so.2 =/lib64/libdl.so.2 ( 0x00007f9be553a000) libc.so.6 =/lib64/libc.so.6 (0x00007f9be51a5000)/lib64/ld-linux-x86-64.so.2 ( 0x00007f9be5966000)

Unfortunately, I stupidly overlooked the answer right there staring me in the face as I am focuses on, "are this library 3 2 PR + bit? ":

' glibc_2.14 ' not found (required by./libhadoop.so.1.0.0)

So, lesson learned. Anyway, the rest at least led me to being able to suppress the warning. So I continued and do everything recommended in the other answers to provide the library path using the Hadoop_opts envir Onment variable to no avail. So I looked at the source code. The module that generates the error tells you the hint (util. Nativecodeloader):

15/06/18 18:59:23 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable

So, off-to-here-see-what it does:

http://grepcode.com/file/repo1.maven.org/maven2/com.ning/metrics.action/0.2.6/org/apache/hadoop/util/ nativecodeloader.java/

Ah, there is some debug level Logging-let's turn that on a see if we get some additional help. This is do by adding the following line to $HADOOP _conf_dir/log4j.properties file:

Log4j.logger.org.apache.hadoop.util.nativecodeloader=debug

Then I-ran a command that generates the original warning, like stop-dfs.sh, and got this goodie:

15/06/18 19:05:19 DEBUG util. nativecodeloader:failed to load Native-hadoop with Error:java.lang.UnsatisfiedLinkError:/opt/hadoop/lib/native/ libhadoop.so.1.0.0:/lib64/libc.so.6:version ' glibc_2.14 ' not found (required by/opt/hadoop/lib/native/ libhadoop.so.1.0.0)

And the answer is revealed in this snippet of the debug message (the same thing, the previous LDD command ' tried ' to T ell me:

' glibc_2.14 ' not found (required by opt/hadoop/lib/native/libhadoop.so.1.0.0)

What version of GLIBC does I have? Here's simple trick-to-find out:

[[email protected] hadoop]$ ldd--versionldd (GNU libc) 2.12

So, can ' t update my OS to 2.14. Only solution are to build the native libraries from sources on my OS or suppress the warning and just ignore it for now. I opted to just suppress the annoying warning for today (but does plan to build from sources on the future) buy using the same Logging options we used to get the debug message, except now, just make it ERROR level.

Log4j.logger.org.apache.hadoop.util.nativecodeloader=error

I Hope this helps others see that a big benefit of the open source software is so can figure this stuff out if you take Some simple logical steps.

Hadoop "Unable to load Native-hadoop library for Y