. Problems and countermeasures caused by five objects heap of net traps

One of the strangest problems we've encountered during development is that when the software loads a lot of large data, there are occasional outofmemoryexception exceptions, but there's a lot more memory available through the memory checker. We suspect that the total amount of available memory is sufficient, but there is not enough contiguous memory-that is, there are a lot of unallocated memory gaps. But not that. NET runtime's garbage collector compresses the memory that is in use, so that the amount of memory that has been freed is connected to one piece? So I went deep into the garbage collection-related content and finally made it clear that the problem was the use of the large object heap (LOH). If you have experienced similar problems or are interested in the details, read on.

If there are no special instructions, the following statements are for 32-bit systems.

First, let's look at another question: regardless of the use of unmanaged memory, in the worst case scenario, what is the amount of memory that is available when the system has a OutOfMemoryException exception? 2G? 1G? 500M? 50M? Or even smaller (do you think I'm joking)? Look at the code below (refer to https://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/).

 public class Program {static void Main (string[] args) {var smallblocksize = 90000;   
         var largeblocksize = 1 << 24;   
         var count = 0;   
         var bigblock = new Byte[0];   
             try {var smallblocks = new list<byte[]> (); while (true) {GC.   
                 Collect ();   
                 Bigblock = new Byte[largeblocksize];   
                 largeblocksize++;   
                 Smallblocks.add (new byte[smallblocksize]);   
             count++;   
             } catch (OutOfMemoryException) {bigblock = null; Gc.   
             Collect ();   
         Console.WriteLine ("{0} Mb allocated", (count * smallblocksize)/(1024 * 1024));   
     } console.readline (); }   
 }

This code constantly alternates between a smaller array and a larger array, where the size of the smaller array is 90, 000 bytes, and the size of the larger array begins at 16M bytes, one byte at a time. As shown in line 15th of the code, in each loop Bigblock references the newly allocated large array, making the previous large array an object that can be garbage collected. When outofmemoryexception occurs, the code actually has the Count decimal group and a large array of size 16M + count in a valid state. The last code outputs the total amount of memory that the decimal group occupies when the exception occurs.

Here is the result of the operation on my machine--how much different from your predictions? To remind you, if you want to test this code yourself, and your machine is 64 bits, make sure to change the build target to x86.

Mb Allocated

Considering that 32-bit program has 2G of available memory, the usage rate here is only 1%!

Here is the reason for this. To be clear, I just want to clarify the problem in the simplest way, so some languages may not be accurate and can refer to http://msdn.microsoft.com/en-us/magazine/cc534993.aspx for a more detailed description.

. NET garbage collection mechanism is based on the concept of "Generation", and altogether there are G0, G1, G2 three Generation. In general, each newly created object belongs to G0, and the object goes to the next generation (G0-> G1->) Each time it undergoes a garbage collection process and is not reclaimed, but it is still in G2 if the object is already in G2.

When the software starts to run, the runtime reserves a contiguous memory for each generation (this is not strictly true, but does not affect the description of the problem), while maintaining a pointer p that points to the unused portion of the memory area, returning directly to the address where P resides when you need to allocate space for the object. and adjust the p accordingly, as shown in the following figure. "By the way, it's precisely because of this technology that creating an object in. NET is much faster than creating objects in the C or C + + heap--and, of course, when the latter does not use additional memory management modules. 】