Use of out-of-heap memory in Java, things to remember about memory recycling and unresolved legacy issues (such as the Big God solution)

the JVM can use 2 different kinds of memory: heap memory and out-of-heap memory, heap memory is fully allocated and freed by the JVM, and if the program does not have defective code causing memory leaks, then you will not encounter java.lang.OutOfMemoryError this error. The use of out-of-heap memory is designed to directly allocate and free memory for increased efficiency. After JDK5.0, there are 2 ways in which you can directly manipulate local memory in your code: using undisclosed unsafe and NIO packets under Bytebuffer.

For an introduction to unsafe objects and how to get them, you can refer to my other blog. Java Gets an instance of the unsafe class and cancels the Eclipse compilation error

Using bytebuffer to allocate local memory is very simple, just bytebuffer.allocatedirect (10 * 1024 * 1024).

C Language memory allocation and release function Malloc/free, must be corresponding, otherwise there will be a memory leak or a wild pointer illegal access. Do we need to manually release the acquired out-of-heap memory in Java?

1. First we look at the Bytebuffer provided in NiO

Import Java.nio.bytebuffer;public class testdirectbytebuffer{//-xx:maxdirectmemorysize=40mpublic static void Main ( String[] args) throws Exception{while (true) {<pre name= "code" class= "java" ><span style= "White-space:pre" > </span>      Bytebuffer buffer = bytebuffer.allocatedirect (10 * 1024 * 1024);

}}}

we set the maximum heap outside memory to 40M, run this code will find: The program can continue to run, will not report OutOfMemoryError. If you use-verbose:gc-xx:+printgcdetails, you will find that the program is frequently garbage collected activities. So we can come to the conclusion that Bytebuffer.allocatedirect allocated out-of-heap memory does not need to be released manually, and there is no manual release API available in Bytebuffer. That is to say, use bytebuffer without worrying about the release of heap memory, unless the heap memoryThe Bytebuffer object has a memory leak because of an error encoding.

2. Next we look at the effect of using unsafe directly in the method body

Import Sun.misc.unsafe;public class testunsafememo{//-xx:maxdirectmemorysize=40mpublic static void Main (string[] args ) throws Exception{unsafe Unsafe = Getusafeinstance.getunsafeinstance (); while (true) {long pointer = Unsafe.allocatememory (1024 * 1024 * 20); SYSTEM.OUT.PRINTLN (Unsafe.getbyte (pointer + 1));//If the memory is not freed, the run time will be error java.lang.outofmemoryerror//unsafe.freememory (pointer);}}}

This program will report OutOfMemoryError errors, that is, allocatememory and freememory, equivalent to malloc and free in C voice, must manually release the allocated memory .

3, similar to Bytebuffer, the unsafe allocated memory encapsulated in a class

Import Sun.misc.unsafe;public class objectinheap{private Long address = 0;private unsafe unsafe = getusafeinstance.getuns Afeinstance ();p ublic objectinheap () {address = Unsafe.allocatememory (2 * 1024 * 1024);} Exception in thread ' main ' java.lang.OutOfMemoryErrorpublic static void Main (string[] args) {while (true) {Objectinheap heap = new Objectinheap (); System.out.println ("Memory address=" + heap.address);}}

This code will throw outofmemoryerror. This is because the Objectinheap object is allocated in heap memory, and when the object is garbage collected, it does not free out-of-heap memory, because the out-of-heap memory obtained by using unsafe must be released by the program, and the JVM will not help us with this thing . Thus, the use of unsafe is risky and can easily lead to memory leaks.

4. Properly release the out-of-heap memory allocated by unsafe

Although there is a memory leak in the objectinheap of the 3rd case, the design of this class is reasonable, it encapsulates the direct memory well, and the caller of this class does not feel the presence of direct memory. How to solve the memory leak problem in objectinheap? You can overwrite object.finalize () to invoke the finalize of the object when the object in the heap is about to be released by the garbage collector . Since the JVM only helps us manage memory resources and does not help us manage resources such as database connections, file handles, and so on, we need to release resources in Finalize ourselves.

Import Sun.misc.unsafe;public class revisedobjectinheap{private Long address = 0;private unsafe unsafe = getusafeinstance . Getunsafeinstance ()//Allow the object to occupy heap memory, trigger [full gcprivate byte[] bytes = Null;public revisedobjectinheap () {address = Unsafe.allocatememory (2 * 1024x768); bytes = new byte[1024 * 1024];} @Overrideprotected void Finalize () throws Throwable{super.finalize (); System.out.println ("Finalize." + bytes.length); unsafe.freememory (address);} public static void Main (string[] args) {while (true) {revisedobjectinheap heap = new Revisedobjectinheap (); System.out.println ("Memory address=" + heap.address);}}

We override the Finalize method to manually release the allocated out-of-heap memory . If the objects in the heap are recycled, then the corresponding amount of out-of-heap memory is also freed. Here's one thing to keep in mind :

Let the object occupy heap memory, trigger [full gcprivate byte[] bytes = NULL;

The main purpose of this line of code is to trigger the garbage collection behavior of heap memory, which in passing executes the finalize of the object to free out of heap memory. If there is not this line of code or the allocated byte array is small, the program will still report OutOfMemoryError after a period of time . This is because the heap memory used is small (about dozens of bytes) each time a 1 Revisedobjectinheap object is created, but it consumes 2M of out-of-heap memory. Such a heap of memory is sufficient (in which case garbage collection of heap memory is not performed), but the out-of-heap memory is insufficient, so OutOfMemoryError is not reported .

Although the improved REVISEDOBJECTINHEAP does not have an out-of-heap memory leak, this workaround wastes heap memory unnecessarily. A simple look at the next Bytebuffer source, its internal allocation of external memory is also through the unsafe.allocatememory () implementation. How does the Bytebuffer implement the out-of-heap memory release? is it through the 4th kind of similar revisedobjectinheap practice? Welcome to the Great God to guide the maze Ah!

Use of out-of-heap memory in Java, things to remember about memory recycling and unresolved legacy issues (such as the Big God solution)