linux 靜態連結 mysql glibc 庫的悲催過程 – mango的日誌 – 網易部落格

linux 靜態連結 mysql glibc 庫的悲催過程 - mango的日誌 - 網易部落格

linux 靜態連結 mysql glibc 庫的悲催過程   

2012-12-29 19:09:31|  分類： 預設分類 |  標籤：linux  靜態連結  static  linking  glibc   |字型大小 大中小 訂閱

這兩天需要把項目做成靜態連結版本，便於在各公司的各種linux平台上運行。

我想這還不簡單，連結參數加一個-static不就行了，但接下來解決一系列問題的時間遠遠超出我的意料

開發環境：

CentOS release 5.8 (Final)

gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-52)

問題是由於靜態連結mysql庫引起的，我連結mysql庫的參數是這樣寫的：

LIB_MYSQL     = $(shell mysql_config --libs)

發現靜態連結悲劇了，好幾百個連結錯誤，都是undefined，需要krb5庫。連結加上krb5庫：

LIB_KBR5     = $(shell krb5-config --libs)

最終部分連結參數是這樣的：$(LIB_MYSQL) $(LIB_KBR5) $(LIB_THREAD) -static

結果連結器還是報錯：

/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_get_principal':
(.text+0xfaa): undefined reference to `keyctl_read_alloc'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_getkeycount':
(.text+0x1129): undefined reference to `keyctl_read'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_clearcache':
(.text+0x1194): undefined reference to `keyctl_clear'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_clearcache':
(.text+0x11c1): undefined reference to `keyctl_clear'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x1692): undefined reference to `request_key'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x16bb): undefined reference to `keyctl_read'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x170c): undefined reference to `keyctl_search'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x1733): undefined reference to `add_key'

......

接下來就是排查是不是還缺什麼庫啊，缺什麼連結時就加什麼，結果都沒能把undefined完全消滅。很鬱悶，就靜態連結mysql庫，這都搞不定？

折騰了很久，還是把問題描述給一個朋友聽，經他提醒，才想到是不是我用yum自動安裝mysql的問題，要不自己下載mysql源碼包，手動編譯安裝試試？下載了mysql-5.5.29.tar.gz，經過tar -xvf，cmake，make，make install，並且把連結mysql的連結參數手動指定了一下：
LIB_MYSQL     = /usr/local/mysql/lib/libmysqlclient.a

哈哈，的確不需要連結krb5的庫了，但是有幾個新的undefined，但是還好，只有幾個，而且清晰：

/usr/local/mysql/lib/libmysqlclient.a(client_plugin.c.o): In function `add_plugin':
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:178: undefined reference to `dlclose'
/usr/local/mysql/lib/libmysqlclient.a(client_plugin.c.o): In function `mysql_load_plugin_v':
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:354: undefined reference to `dlopen'
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:382: undefined reference to `dlsym'
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:374: undefined reference to `dlerror'
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:385: undefined reference to `dlclose'

於是，連結的庫增加libdl.a(屬於glibc裡的)，部分連結參數如下：

-lcrypt -L/usr/local/mysql/lib/ -lmysqlclient -ldl -lpthread -static

哦也，終於連結過了！雖然有幾個讓人討厭的warning:

/usr/local/mysql/lib//libmysqlclient.a(client_plugin.c.o): In function `mysql_load_plugin_v':
/home/win/mysql-5.5.29/sql-common/client_plugin.c:354: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(mf_pack.c.o): In function `expand_tilde':
/home/win/mysql-5.5.29/mysys/mf_pack.c:387: warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(libmysql.c.o): In function `read_user_name':
/home/win/mysql-5.5.29/libmysql/libmysql.c:422: warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(mf_pack.c.o): In function `expand_tilde':
/home/win/mysql-5.5.29/mysys/mf_pack.c:389: warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(client.c.o): In function `mysql_real_connect':
/home/win/mysql-5.5.29/sql-common/client.c:3247: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-redhat-linux/4.4.6/../../../../lib64/libpthread.a(libpthread.o): In function `sem_open':
(.text+0x764d): warning: the use of `mktemp' is dangerous, better use `mkstemp'
/usr/local/mysql/lib//libmysqlclient.a(libmysql.c.o): In function `mysql_server_init':
/home/win/mysql-5.5.29/libmysql/libmysql.c:153: warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

於是很激動的啟進程，剛起來，就宕掉了！！！！用GDB起來，發現是宕在調用libdl的庫裡。

其實上面的7個warning中，後面6個我認識的：在以前的另一個項目中(環境是ubuntu server)，也是靜態連結，一直都有後面這6個warning，但是這幾個函數沒有用到，所以進程沒有問題，當時項目進度又異常的緊，大家就一直沒有花心思去研究消掉這幾個warning。現在新增加的第一個warning，其中dlopen會調用到，會造成宕機，這下無法逃避，所以我接下來的精力就花在消除這幾個warning上。

就是這麼一個問題：Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking，這是一個很常見的問題：

static linking issue with glibc

但是我各種關鍵字組合google搜尋，發現問這個問題的人不少，討論也很熱烈，但是都沒有人給出問題的解決方案，下面兩個串連是比較明確的：

http://bytes.com/topic/c/answers/877929-lnk-error-using-dlopen-statically-linked-apps-requires-runtime-shared-lib

https://bugzilla.redhat.com/show_bug.cgi?id=111298

http://www.qtcentre.org/threads/32521-Problems-with-static-linking-gt-segmentation-fault

其中有個老外說了下面一段：

Jakub Jelinek 2003-12-01 17:49:41 EST

If a RHL9 built statically linked binary using NSS works on older  glibcs, then you are just lucky, nothing else.  It certainly will not  work on all older glibcs, only on some and only under some circumstances.  Statically linked programs which are not self-contained and depend  on the installed glibc (be it because of using NSS or iconv (which  both dlopen glibc internally), using dlopen directly or even using  locale support) are really the least portable thing you can build.    Dynamically linking is uncomparably more portable.  You can link some libraries statically using -Bstatic -lfoo -lbar -Bdynamic,  yet link against the crucial system libraries dynamically.    The warnings were added so that everyone is aware of the problems  with -static.  For statically linked binaries using NSS/iconv/dlopen  the dynamic linker needs to be compiled into the statically linked  program, yet the shared libraries it loads come from a different  glibc if the statically linked binary is moved onto a different system.  This means the glibc private interface between the dynamic linker  and libc.so/libpthread.so is exposed, but this interface is constantly  changing (and has been changing for years in the past as well).    So, the only supported way of using NSS/iconv/dlopen in statically  linked programs is if they are compiled/linked against the same glibc  as they are run with.

 搜尋可以開啟這個頁面：

http://www.google.com.hk/search?hl=en&source=hp&q=Using+'gethostbyname'+in+statically+linked+applications+requires+at+runtime+the+shared+libraries+from+the+glibc+version+used+for+linking&aq=f&oq=&aqi=

不過都沒能解決問題，warning依然存在。朋友還提醒，是不是系統版本裝的有問題？或者開發環境裝的有問題？好吧，我已經沒有更好的方法了，還讓IT的同事重新找電腦裝了新環境CentOS6.3 DVD版，折騰了很久，編譯，連結，結果還是那7個warning，它們就死死的杵在那裡！

最終，這個問題都沒能解決。但是項目得繼續，所以最終採用了一個折中方案：

mysql和另一些自有庫靜態連結，glibc,libstdc++等一些常見的基礎庫動態連結。

-Wl,-dn 其它自己的靜態庫 -L/usr/local/mysql/lib/ -lmysqlclient -Wl,-dy -lpthread -ldl -lcrypt

據說，這才是標準做法~~ 因為網上有人反問：為什麼要靜態連結glibc庫？

當然，也可以直接在編譯階段將靜態庫編進項目裡，例如：

g++ main.cpp libmysqlclient.a