Record exception tuning during tomcat thread creation: unable to create new native thread, tomcat Tuning

Record exception tuning during tomcat thread creation: unable to create new native thread, tomcat Tuning

During a one-time test, the following exception occurs when 300 concurrent requests are detected:

HTTP Status 500-Handler processing failed; nested exception is java. lang. OutOfMemoryError: unable to create new native thread

The exception was a bit flustered. After all, the number of concurrent programs was small, and suddenly such an exception was a bit hard to find. But in any case, search for the reason first.

This error is caused by the failure to create a new thread, because there may be no more space for creating a thread, and there is also a formula for calculating:

(MaxProcessMemory-JVMMemory-ReservedOsMemory)/(ThreadStackSize) = Number of threads

The parameters here mean:
MaxProcessMemory: Maximum addressing space of a process.
JVMMMEMORY: jvm memory space (heap + permanent zone)-Xmx size (should be the actual allocated size)
ReservedOsMemory: Memory reserved by the operating system
ThreadStackSize:-Xss size

Too lazy to write, find a reference: https://my.oschina.net/xinxingegeya/blog/744462

So I will check the parameters of the liunx system:

[root@RHEL63temp ~]# ulimit -acore file size          (blocks, -c) 0data seg size           (kbytes, -d) unlimitedscheduling priority             (-e) 0file size               (blocks, -f) unlimitedpending signals                 (-i) 63628max locked memory       (kbytes, -l) 64max memory size         (kbytes, -m) unlimitedopen files                      (-n) 1024pipe size            (512 bytes, -p) 8POSIX message queues     (bytes, -q) 819200real-time priority              (-r) 0stack size              (kbytes, -s) 10240cpu time               (seconds, -t) unlimitedmax user processes              (-u) 1024virtual memory          (kbytes, -v) unlimitedfile locks                      (-x) unlimited

There are only 1024 max user processes among them, thinking that the problem may not be the limitation of creation, but why is it necessary to create such a multi-thread? After all, there are only 300 concurrent threads, so tomcat can use up to 300 threads to process requests?

So I thought about the program code, but I still need to find the cause from the code. So I temporarily started to study the JVisualVM monitoring tool and made configuration on the server. There are tutorials on the Internet. Because I am using Tomcat, I will directly monitor Tomcat and add some parameters in catalina. sh:

JAVA_OPTS="-server -Xmx384m -Xms128m -XX:PermSize=128M -XX:MaxPermSize=256m"-Dcom.sun.management.jmxremote.port=9998 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=192.168.49.199"

In this way, you can use JVisualVM to monitor through JMX. After the connection, the cause of the problem is found.

There are two main threads:
1. tomcat itself needs to support concurrent threads
2. A large number of threads generated by smack, and continuous stress testing will find that the threads of smack are not released.

The key here is the use of smack, because the system implements a function that initiates ajax requests through the web page, and then simulates the instant sending of messages through the user on the server. Because 300 requests are concurrently sent, each request must create a smack connection, while smack is a library used for client development, after startup, about three threads will be created to connect to and process server communication. This leads to 300*3 threads at the same time, so the number of threads created by the club is full during concurrency.

Since the cause of the problem is found, the problem may still be the use of smack. After all, smack is a client library and is not suitable for this server scenario.

The solution is to use other methods to send messages instead of smack, so that only a small number of threads can be created to meet the requirements, and the processing speed is greatly improved.

 

Note: This article is original. You are welcome to repost it. Please provide this article link clearly on the article page! If you think this article is good, please click the recommendation in the lower right corner. Thank you very much! Http://www.cnblogs.com/5207