Use Out-of-heap memory in java, precautions for memory recovery and unresolved issues (etc)

Use Out-of-heap memory in java, precautions for memory recovery and unresolved issues (etc)

JVM can use two types of memory: heap memory and off-heap memory. The JVM is responsible for allocating and releasing the heap memory. If the program has no defect code, the memory will be leaked, so we will not encounter java. lang. outOfMemoryError. The out-of-heap memory is used to directly allocate and release memory and improve efficiency. After JDK, two methods can be used to directly operate the local memory in the Code: Unsafe and ByteBuffer In the NIO package.

For details about Unsafe objects and how to obtain them, refer to my other blog.Java obtains the Unsafe class instance and cancels the eclipse compilation error.

It is very easy to use ByteBuffer to allocate local memory. You can directly use ByteBuffer. allocateDirect (10*1024*1024.

The memory allocation and release function malloc/free in C language must correspond to each other. Otherwise, memory leakage or unauthorized access to the wild pointer may occur. In java, do we need to manually release the obtained off-heap memory?

1. First, let's take a look at the ByteBuffer provided by NIO.

Import java. nio. ByteBuffer; public class TestDirectByteBuffer {//-XX: MaxDirectMemorySize = 40 Mpublic static void main (String [] args) throws Exception {while (true ){
      ByteBuffer buffer = ByteBuffer.allocateDirect(10 * 1024 * 1024);
}}}
We set the maximum off-heap memory to 40 MB. When running this code, we will find that the program can continue running without reporting OutOfMemoryError. If-verbose: gc-XX: + PrintGCDetails is used, the program will frequently perform garbage collection. So we can draw a conclusion:The off-heap memory allocated by ByteBuffer. allocateDirect does not need to be manually released, and no API for manual release is provided in ByteBuffer.. That is to say, using ByteBuffer does not have to worry about releasing off-heap memory unless the ByteBuffer object in the heap memory is leaked due to incorrect encoding.
2. Next let's take a look at the effect of using Unsafe directly in the method body.
Import sun. misc. unsafe; public class TestUnsafeMemo {//-XX: MaxDirectMemorySize = 40 Mpublic static void main (String [] args) throws Exception {Unsafe unsafe = GetUsafeInstance. getUnsafeInstance (); while (true) {long pointer = unsafe. allocateMemory (1024*1024*20); System. out. println (unsafe. getByte (pointer + 1); // if the memory is not released, an error occurs during a period of time. lang. outOfMemoryError // unsafe. freeMemory (pointer );}}}
This program will report an OutOfMemoryError, that isAllocateMemory and freeMemory are equivalentMalloc and free. You must manually release the allocated memory..

3. Similar to ByteBuffer, Unsafe memory allocation is encapsulated into a class.
import sun.misc.Unsafe;public class ObjectInHeap{private long address = 0;private Unsafe unsafe = GetUsafeInstance.getUnsafeInstance();public ObjectInHeap(){address = unsafe.allocateMemory(2 * 1024 * 1024);}// Exception in thread "main" java.lang.OutOfMemoryErrorpublic static void main(String[] args){while (true){ObjectInHeap heap = new ObjectInHeap();System.out.println("memory address=" + heap.address);}}}
This code will throw an OutOfMemoryError. This is because the ObjectInHeap object is allocated in the heap memory. When the object is reclaimed by garbage collection, the off-heap memory is not released becauseThe off-heap memory obtained using Unsafe must be released by the program. The JVM will not help us with this.. It can be seen that the use of Unsafe is risky and can easily lead to memory leakage.

4. Properly release the Unsafe allocated off-heap memory
Although the ObjectInHeap Memory leakage exists in 3rd cases, the design of this class is reasonable. It encapsulates direct memory, and callers of this class cannot feel the existence of direct memory. So how can we solve the problem of Memory leakage in ObjectInHeap? You can override Object. finalize (). When the Object in the heap is about to be released by the garbage collector, the finalize of the Object will be called. Since JVM only helps us manage memory resources and does not help us manage database connections, file handles, and other resources, we need to release resources on finalize.
Import sun. misc. unsafe; public class RevisedObjectInHeap {private long address = 0; private Unsafe unsafe = GetUsafeInstance. getUnsafeInstance (); // Let the object occupy the heap memory, trigger [Full GCprivate byte [] bytes = null; public RevisedObjectInHeap () {address = unsafe. allocateMemory (2*1024*1024); bytes = new byte [1024*1024];} @ Overrideprotected void finalize () throws Throwable {super. finalize (); System. out. println ("finalize. "+ bytes. length); unsafe. freeMemory (address);} public static void main (String [] args) {while (true) {RevisedObjectInHeap heap = new RevisedObjectInHeap (); System. out. println ("memory address =" + heap. address );}}}
WeOverwrite the finalize method and manually release the allocated off-heap memory.. If the objects in the heap are recycled, the occupied off-heap memory is also released.Note the following::
// Let the object occupy the heap memory and trigger [Full GCprivate byte [] bytes = null;
The main purpose of this line of code is to trigger the garbage collection behavior of the heap memory and release the off-heap memory along with the finalize of the execution object. If this line of code is not available or the allocated byte array is small, an OutOfMemoryError will still be reported after the program runs for a period of time. This is because every time a RevisedObjectInHeap object is created, the occupied heap memory is very small (about dozens of bytes), but it takes 2 MB of off-heap memory. In this way, the heap memory is sufficient (in this case, garbage collection of heap memory is not executed), but the off-heap memory is insufficient, so no OutOfMemoryError is reported.

Although the improved RevisedObjectInHeap does not expose off-heap memory, this solution wastes the heap memory for no reason. A simple look at the ByteBuffer source code, it is internalAllocating off-heap memory is also implemented through unsafe. allocateMemory. ThatHow does ByteBuffer release off-heap memory?Is there 4th similar RevisedObjectInHeap practices? Thank you!