標籤:loop nat 資源 超出 strong imu except rar inux
不是全部的OutofMemory exception都是記憶體問題。。。
前幾天有個客戶的site報了下面錯誤:
[ERROR][thread ] Could not start thread Timer-72025. Resource temporarily unavailableException in thread "Timer-72024" java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1097).Attempting to allocate 2G bytesThere is insufficient native memory for the JavaRuntime Environment to continue.Possible reasons: The system is out of physical RAM or swap space In 32 bit mode, the process size limit was hitPossible solutions: Reduce memory load on the system Increase physical memory or swap space Check if swap backing store is full Use 64 bit Java on a 64 bit OS Decrease Java heap size (-Xmx/-Xms) Decrease number of Java threads Decrease Java thread stack sizes (-Xss) Disable compressed references (-XXcompressedRefs=false) at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at java.util.Timer.<init>(Timer.java:137) at java.util.Timer.<init>(Timer.java:106) at ...... at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462)
第一反應以為是記憶體不足,於是dump了heap回來看。發現系統僅僅佔用了80M的記憶體,遠遠系統記憶體綽綽有餘,那麼為啥會出現這樣的情況呢?查看了出錯的代碼。發現代碼啟動了非常多的線程,遂想起曾經遇到過類似的問題,weblogic裡面啟動的線程太多。超出了linux的最大限制,也是會出這個看起來記憶體流失的日誌。查看系統的限制:
[email protected] limits.d # ulimit -acore file size (blocks, -c) 0data seg size (kbytes, -d) unlimitedscheduling priority (-e) 0file size (blocks, -f) unlimitedpending signals (-i) 38610max locked memory (kbytes, -l) 64max memory size (kbytes, -m) unlimitedopen files (-n) 8192pipe size (512 bytes, -p) 8POSIX message queues (bytes, -q) 819200real-time priority (-r) 0stack size (kbytes, -s) 10240cpu time (seconds, -t) unlimitedmax user processes (-u) 1024virtual memory (kbytes, -v) unlimitedfile locks (-x) unlimited
發現最大的進程數是1024,注意:linux裡面計算線程和進程是用一樣計算的,這裡說的max user processes是包含線程的。
使用下面代碼重現問題:
public class ThreadExample { public static void main(String[] args){ System.out.println(Thread.currentThread().getName()); for(int i=0; i< 2048; i++){ new Thread("" + i){ public void run(){ System.out.println(Thread.currentThread().getName() + " running"); try { Thread.sleep(60000); } catch (InterruptedException e) { e.printStackTrace(); } System.out.println(Thread.currentThread().getName() + " ending"); } }.start(); } }}
運行和輸出:
[email protected]_Z1 etmdaha # su -p occas -c "/opt/jrockit-jdk/bin/java ThreadExample"Main Thread0 running5 running//836 running[ERROR][thread ] Could not start thread 837. Resource temporarily unavailableException in thread "Main Thread" java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1096).Attempting to allocate 3G bytesThere is insufficient native memory for the JavaRuntime Environment to continue.Possible reasons: The system is out of physical RAM or swap space In 32 bit mode, the process size limit was hitPossible solutions: <strong>Reduce memory load on the system Increase physical memory or swap space Check if swap backing store is full Use 64 bit Java on a 64 bit OS Decrease Java heap size (-Xmx/-Xms) Decrease number of Java threads Decrease Java thread stack sizes (-Xss) Disable compressed references (-XXcompressedRefs=false)</strong> at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at ThreadExample.main(ThreadExample.java:6)
linux系統資源有各種限制:記憶體限制。棧限制,檔案數目限制,線程限制。要小心陷進這些問題。
# ulimit -acore file size (blocks, -c) 0data seg size (kbytes, -d) unlimitedscheduling priority (-e) 0file size (blocks, -f) unlimitedpending signals (-i) 79010max locked memory (kbytes, -l) 64max memory size (kbytes, -m) unlimitedopen files (-n) 1024pipe size (512 bytes, -p) 8POSIX message queues (bytes, -q) 819200real-time priority (-r) 0stack size (kbytes, -s) 10240cpu time (seconds, -t) unlimitedmax user processes (-u) 1024virtual memory (kbytes, -v) unlimitedfile locks (-x) unlimited
這裡有兩篇非常好的文章:
http://people.redhat.com/alikins/system_tuning.html#threads
http://kavassalis.com/2011/03/linux-and-the-maximum-number-of-processes-threads/
Weblogic OutOfMemory exception的誤解 -- thread limitation