SSH internal Overflow

Believe that there are some Java development experience, more or less people will encounter outofmemoryerror problems, this problem has plagued me for a long time, with the solution of the accumulation of various problems and the root causes of the exploration, finally has a more in-depth understanding.

Before solving the Java memory overflow problem, you need to have a certain understanding of the memory management of the JVM (Java Virtual machine). The memory that the JVM manages roughly includes three different types of memory areas: Permanent Generation Space (Permanent save area), heap space (heap area), Java Stacks (Java stack). Where the permanent save area mainly holds the class (class) and meta information, class is first loaded into the PermGen space area, the content of the class needs to be stored mainly including methods and static properties. The heap area is used to hold instances of class (that is, objects), and the content that the object needs to store is primarily non-static properties. Each time an object instance is created with new, the object instance is stored in the heap area, which is also managed by the JVM's garbage collection mechanism. The Java stack is similar to most programming languages, including assembly language stack functions, the main basic type variables and the input and output parameters of the method. There is a separate stack in each thread of the Java program. Memory spaces prone to memory overflow problems include: Permanent Generation space and heap space.

The first kind of outofmemoryerror:permgen space

The original intention of this problem is that the program uses a large number of jar or class, so that the Java Virtual machine load class space is not enough, and permanent Generation space. There are two ways to address this type of problem:

1. Increase the size of the xx:permsize and xx:maxpermsize parameters in the Java Virtual machine, where xx:permsize is the initial permanent save area size, and xx:maxpermsize is the maximum permanent save area size. For tomcat6.0, add a line to the end of the list of environment variable names in the catalina.sh or Catalina.bat file (approximately 70 rows): JAVA_OPTS=" -XX:PermSize=64M -XX:MaxPermSize=128m" If Windows Server can also be set in the system environment variable. This memory overflow error can easily occur when using Tomcat to publish Sprint+struts+hibernate schema programs. Using the above method, I successfully solved the problem of frequent outages of the Tomcat server that deployed the SSH project.
2. Cleans the jar under Web-inf/lib in the application, if Tomcat deploys multiple applications, many applications use the same jar, and the common jar can be moved to the Tomcat common Lib, reducing the class's repeated loading. This method is recommended by some people on the Internet, I have not tried, but feel less space, the most reliable is the first method.

The second type of Outofmemoryerror:java heap space

This problem occurs because there are too many objects created by the Java Virtual machine, and the virtual machine allocates enough memory space to the heap for garbage collection, which is related to heap space. There are two ways to solve this kind of problem:

1. Check the program to see if there are dead loops or unnecessarily repeatedly creating a large number of objects. After you find the cause, modify the program and algorithm. I used to write a K-means text clustering algorithm for tens of thousands of of text records (the eigenvector of each record about 10 or so) for text clustering, due to procedural details of the problem, led to the Java heap space memory overflow problem, later through the modification program was resolved.
2. Increase the size of the XMS (initial heap size) and XMX (maximum heap size) parameters in the Java Virtual machine. Such as:set JAVA_OPTS= -Xms256m -Xmx1024m

Third type outofmemoryerror:unable to create new native thread

In Java applications, there are times when such errors occur: Outofmemoryerror:unable to create new native thread. This strange thing is because the JVM has been allocated a lot of memory (such as 1.5G) by the system, And it consumes at least half of the available memory. It has been found that the more memory you allocate to the JVM, the greater the likelihood that the above error will occur, given the large number of threads.

So what is the cause of this problem?

Each 32-bit process can use up to 2G of available memory, because another 2G is reserved by the operating system. This assumes that 1.5G is used for the JVM, and then the remaining 500M of available memory. This 500M part of the memory must be used to load the system DLL, then the real left is perhaps only 400M, and now the key point arises: When you use Java to create a thread, in the JVM's memory will also create a thread object, But it also creates a real physical thread in the operating system (referencing the JVM specification), and the operating system creates the physical thread in the remaining 400 megabytes of memory, rather than in the 1500M memory heap of the JVM. In jdk1.4, the default stack size is 256KB, but in jdk1.5, the default stack size is 1M per thread, so we can create up to 400 available threads in the remaining 400M of available memory.

The conclusion is that to create more threads, you must reduce the maximum memory allocated to the JVM. Another option is to have the JVM host inside your JNI code.

Give an estimate formula for the maximum number of threads that can be created:

(maxprocessmemory-jvmmemory-reservedosmemory)/(Threadstacksize) = number of threads

For jdk1.5, assume that the operating system retains 120M of memory:

1.5GB JVM: (2GB-1.5GB-120MB)/(1MB) = ~380 THREADS1.0GB JVM: (2GB-1.0GB-120MB)/(1MB) = ~880 threads

For jdk1.4 with a stack size of 256KB,

1.5GB allocated to JVM: ~1520 THREADS1.0GB allocated to JVM: ~3520 threads

For this exception we first need to determine, in the event of memory overflow in the process in the end what kind of thread, whether these threads should exist, whether it can be optimized to reduce the number of threads, on the other hand by default Java for each thread allocated stack memory size is 1M, usually, This 1M stack of memory space is sufficient, because in the usual on the stack is only the underlying type of data or object references, these things will not occupy too much memory, we can adjust the JVM parameters, reduce the stack memory allocated to each thread to solve the problem, such as the JVM parameters to add -Xss128k Set the thread stack memory size to 128k

SSH internal Overflow