Java memory overflow Parsing

Core tips: there are many reasons, such as: 1. the data volume is too large; endless loops; too many static variables and static methods; recursion; unable to determine whether the referenced object is being referenced; 2. the virtual machine does not recycle memory (Memory leakage). To put it bluntly, the memory used by the program is larger than the maximum memory provided by the virtual machine, and memory overflow occurs. There are many reasons for memory overflow, such:

1. Too large data volume; endless loops; too many static variables and static methods; recursion; unable to determine whether the referenced object is being referenced;

2. the VM does not recycle memory (Memory leakage );

To put it bluntly, a memory overflow occurs when the memory used by the program is larger than the maximum memory provided by the virtual machine. The memory overflow problem depends on the business and system size. For some systems, memory overflow may be uncommon, but some systems are still very common solutions,

One is to optimize the program code. if the business is huge and the logic is complex, we should minimize the reference of global variables. Releasing the reference when the program uses the variable will allow the Garbage Collector to recycle and release resources.
The second is physical solution, increase the physical memory, and then modify it through:-xms256m-xmx256m-XX: maxnewsize = 256 m-XX: maxpermsize = 256m

I. memory overflow type
1. java. Lang. outofmemoryerror: permgen Space

JVM manages two types of memory, heap and non-heap. Heap is used by developers. It is created at JVM startup. Non-heap is reserved for JVM and used to store class information. Unlike the heap, GC does not release space during runtime. If the web app uses a large number of third-party jar files or the application has too many class files, but the maxpermsize setting is small, exceeding this will also cause excessive memory usage and overflow, or during hot deployment of Tomcat, the Environment loaded above will not be cleaned up, but the context will be changed to a new deployment, so there will be more and more non-heap content.

2. java. Lang. outofmemoryerror: Java heap Space

The first case is a supplement. The main problem is that it appears in this case. The default space (-XMS) is 1/64 of the physical memory, and the maximum space (-xmx) is 1/4 of the physical memory. If the remaining memory is less than 40%, the JVM will increase the heap to the xmx setting value. If the remaining memory exceeds 70%, the JVM will reduce the heap to the XMS setting value. Therefore, the xmx and XMS settings of the server should be set to the same value to avoid adjusting the size of the VM heap after each GC operation. If the physical memory is infinitely large, the maximum JVM memory is related to the operating system. Generally, 32-bit machines are between 1.5 GB and 3 GB, and 64-bit machines are not limited.

Note: If the XMS value exceeds the xmx value, or the sum of the heap maximum value and the non-heap maximum value exceeds the physical memory or the maximum operating system limit, the server cannot be started.

Garbage collection GC role

JVM calls GC frequently. Garbage collection is mainly performed in two cases:

When the application thread is idle, and when the Java memory heap is insufficient, GC is continuously called. If continuous collection fails to solve the problem of insufficient memory heap, an out of memory error is reported. This exception is determined by the system running environment, so it cannot be expected when it will appear.

According to the GC mechanism, program running may cause changes in the system runtime environment and increase the chance of GC triggering.

To avoid these problems, the program design and writing should avoid the memory usage and GC overhead of spam objects. It is shown that calling system. GC () can only be recommended that JVM recycle junk objects in memory, but not immediately,

One is that it cannot solve the problem of insufficient memory resources and increase GC consumption.

Ii. JVM memory zone Composition
Java stack and stack

Java divides memory into two types: stack memory and heap memory.

1. The basic type variables defined in the function and the referenced variables of the object are allocated in the function stack memory;

2. Heap memory is used to store objects and arrays created by new.

When a variable is defined in a function (code block), Java allocates memory space for the variable in the stack. When the scope of the variable is exceeded, java Automatically releases the memory space allocated for the variable. The memory allocated in the heap is managed by the Java Virtual Machine's automatic garbage collector.

The advantage of heap is that the memory size can be dynamically allocated, and the lifetime does not need to be told to the compiler in advance because it dynamically allocates memory at runtime. The disadvantage is that the memory needs to be dynamically allocated at runtime, And the access speed is slow;

The advantage of stack is that the access speed is faster than the heap speed, but the disadvantage is that the data size and lifetime in the stack must be determined inflexible.

Java heap is divided into three areas: new, old, and permanent.

GC has two threads:

The newly created object is allocated to the new area. When the area is filled, it will be moved to the old area by the GC auxiliary thread, when the old area is full, the main GC thread is triggered to traverse all objects in the heap memory. The size of the old area is equal to xmx minus-xmn.

Java stack Storage

Stack adjustment: the parameter is + usedefastackstacksize-xss256k, indicating that each thread can apply for a stack space of kb.

Each thread has its own stack

Iii. How to Set virtual memory in JVM
Tip: if 98% is used for GC and the available heap size is less than 2% in JVM, this exception is thrown.

Tip: the heap size should not exceed 80% of the available physical memory. Generally, you must set the-XMS and-xmx options to the same, and-xmn to the-xmx value of 1/4.

Tip: the initial memory allocated by JVM is specified by-XMS. The default value is 1/64 of the physical memory. The maximum memory allocated by JVM is specified by-xmx. The default value is 1/4 of the physical memory.

By default, when the free heap memory is less than 40%, the JVM will increase the heap until the maximum limit of-xmx. When the free heap memory is greater than 70%, the JVM will reduce the minimum limit of heap until-XMS. Therefore, the server generally sets-XMS and-xmx to be equal to each other to avoid adjusting the heap size after each GC.

Tip: if the physical memory is infinitely large, the maximum JVM memory has a great relationship with the operating system.

Simply put, although the 32-bit processor has a controllable memory space of 4 GB, the specific operating system will impose a limit,

This limit is generally 2 GB-3 GB (1.5 GB-2 GB in windows and 2 GB-3 GB in Linux ), the 64-bit and above processors will not be limited.

Tip: Note: If the XMS value exceeds the xmx value, or the sum of the maximum heap value and the maximum non-heap value exceeds the limit of the physical memory or operating system, the server cannot be started.

Tip: Set newsize and maxnewsize to be equal. The size of "new" should not be greater than half of "old" because if the old area is not large enough to frequently trigger the "Main" GC, greatly reduced performance

JVM uses-XX: permsize to set the non-heap memory initial value. The default value is 1/64 of the physical memory;

Set the maximum non-heap memory size by XX: maxpermsize. The default value is 1/4 of the physical memory.

Solution: manually set heap size

Modify tomcat_home/bin/Catalina. bat

Add the following lines to "Echo" using catalina_base: $ catalina_base:

1. Java_opts = "-server-xms800m-xmx800m-XX: maxnewsize = 256m"

Iv. Use of performance check tools
Locate Memory leakage:

Jprofiler is mainly used to check and track the performance of the system (limited to Java Development. Jprofiler can monitor the system's memory usage and garbage collection and thread running status at any time to monitor the JVM running status and performance.

1. The memory of the application server is used unreasonably for a long time. The memory is often occupied at a high level, which is hard to be recycled to a low level;

2. The application server is extremely unstable and restarts every two days, sometimes even every day;

3. the Application Server often performs full GC (garbage collection), which takes about 30-40 seconds. When performing full GC, the application server does not respond to the customer's transaction request, it has a significant impact on system performance.

Because the development environment and product environment are different, this problem sometimes occurs in the product environment. Generally, you can use tools to track the memory usage of the system, in some cases, a large amount of memory may be used at some time, resulting in out of memory. In this case, we should continue to track whether there will be any decrease in the future,

If it remains high, it is certainly because the program causes memory leakage.

5. Features and solutions for unrobust code
1. Release reference of useless objects as soon as possible. A good way is to enable the reference variable to be automatically set to null after exiting the active domain when using a temporary variable, suggesting that the Garbage Collector collects the object to prevent memory leakage.

For instances with pointers, JVM will not recycle this resource, because garbage collection will take null objects as garbage, improving the efficiency of GC collection mechanism;

2. In our program, a large amount of string processing is inevitable. To avoid the use of string, a large number of stringbuffer should be used. Each string object must occupy an independent area of memory;

1. String STR = "AAA ";
3. String str2 = "BBB ";
5. String str3 = STR + str2; // If STR and str2 are not called after this execution, it will be stored in the memory and waited for Java GC to be recycled, if this happens too many times in the program, the above error will be reported. We recommend that you use stringbuffer when using strings instead of using strings, which saves a lot of overhead;

3. Use as few static variables as possible, because static variables are global and GC will not be recycled;

4. Avoid creating objects in a centralized manner, especially large objects. JVM will suddenly need a large amount of memory, which will inevitably trigger GC to optimize the system memory environment. The declared array space is displayed, and the number of applications is large.

This is a case for your warning:

Jspsmartupload is used for file upload, and Java is often used during running. outofmemoryerror: use the top command to check the process usage and find that the memory is less than 2 MB. It took a long time to find that it was a jspsmartupload problem. Decompile the source code file (class file) of the jspsmartupload component into a Java file:

1. M_totalbytes = m_request.getcontentlength ();
2. M_binarray = new byte [m_totalbytes];

The variable m_totalbytes indicates the total length of the File Uploaded by the user, which is a large number. If you declare a byte array with such a large number and allocate memory space to each element of the array, and the m_binarray array cannot be released immediately, the JVM garbage collection is indeed faulty, the result is a memory overflow.

The reason why jspsmartupload is to do this is that according to the HTTP upload standard of rfc1867, a file stream is obtained without knowing the length of the file stream. The designer only knows the length of the file once after the servletinputstream operation, because no stream knows the size. Only when the length of the file is known can the user limit the length of the file to be uploaded. To save this trouble, the jspsmartupload designer directly opens the file in the memory and determines whether the length complies with the standard, and writes the file to the server's hard disk. This results in memory overflow, which is just my guess.

When programming, do not apply for large space in the memory, because the memory of the Web server is limited, and use stream operations as much as possible, such

Byte [] mfilebody = new byte [1, 512];

Blob vfield = Rs. getblob ("filebody ");

Inputstream instream = vfield. getbinarystream ();

Fileoutputstream Fos = new fileoutputstream (SaveFilePath + cfilename );

Int B;

While (B = instream. Read (mfilebody ))! =-1 ){

FOS. Write (mfilebody, 0, B );

}

FOS. Close ();

Instream. Close ();

5. Try to use the object pool technology to improve system performance. objects with a long life cycle have objects with a short life cycle, which may cause memory leakage, for example, when a large collection object has a large data volume of business objects, you can consider processing them in multiple parts and then solve the policy of releasing one piece.

6. Do not create objects in frequently called methods, especially avoid creating objects in loops. You can use hashtable as appropriate. The vector creates a group of object containers and then retrieves those objects from the containers without dropping them after each new operation.

7. It usually occurs when a large file is opened or too much data is taken with the database at a time, resulting in an out of memory error. In this case, the maximum data volume is calculated, set the minimum and maximum memory space values.

Reference: http://ajava.org/course/java/12437.html