Effective prevention of. NET application OOM experience memo

According to my personal development and system tuning experience, most of the memory leaks are directly related to bad development habits. There are several development experiences that can effectively prevent OOM and I will share them with you.

 

I. Batch and paging

This is a simple, but practical topic.

Each qualified coder must be aware of data processing by page or batch multiple times. Reading or querying result sets with a large amount of data is a large memory footprint and is one of the direct reasons for the decline in system performance.

In typical Internet Web applications, when the data volume is large and high concurrency, a large amount of user data is always retrieved without paging or batch processing, it is easy to cause excessive memory overhead and system memory to be tight. For example, when we perform file operations, we need to consider the size of the file when reading the file content.

If an indiscriminate query still occurs in your project, return N (how big is n ?) Dataset, able, or list record of a record, or query a large amount of data written to a temporary table, or read a large file at a time.

 

Ii. use static data with caution

This is also a common but implicit culprit that causes memory leakage.

Static class, static field, static attribute, static delegate, static method... Of course, the static benefits are obvious, such as convenient calls and resident memory to improve performance.CodeIn addition to the entity layer, the explicit layer (which is terrible to say, I used to be in a web applicationProgramAnd the data access layer, business logic layer, public components, configuration management, and so on can all use static data.

Compared with the frequent insufficient memory caused by big data queries, too many static applications will not leak memory at half past one. As the system runs, there are more and more static things, and the memory overhead increases. Due to the GC recycle policy, the memory occupied by the invalid static state is not easily released in time. Over time, the memory is insufficient.

Static usage is very common in Layered applications, and the hidden risks are not easily exposed due to its advantages. Therefore, many programmers are very persistent in static usage.

 

Third, the second-party library and third-party library are not managed resources. The dispose mode is preferred.

Some applications need to use packaged libraries or third-party libraries, or use unmanaged resources, such as COM components. net Automatic memory management and recovery, many people think that I use it to complete the function.

The actual situation is that you have called someone else's library, and someone else's library is likely to occupy your memory without knowing it.

Every time you call other people's resources, You Should Be alert: You can use your class library, and the usage of my (memory) will not work.

If you are familiar with the automatic memory management, GC, and dispose mode, you will definitely think about the using when calling other people's resources. Or, it is just a little effort to force null, believe that some good programming habits can make automatic memory management more effective. The fear is that many people are taking the initiative and the test is inadequate. After successfully calling the function, they will be OK and hand it over to others.

 

4. Reduce temporary string objects

This is really too familiar, no matter what form of application, where can the character string be lost?

We can see that they think of Concatenated strings, resident pools, and so on.

Yes, the improper use of string operations will also cause memory insufficiency exceptions, and this is not a sensation.

For example, for string + =, this operation is endless in many applications. We all know that the + = operation will cause many temporary string objects. These objects are easy to occupy memory space due to CLR's resident processing of strings. For highly concurrent web applications, string operations can be seen everywhere, and the length of the string is uncertain, the front-end page is spliced in a variety of ways. Over time, memory usage is a major problem. The CLR optimizes the string so that the string is not recycled first. If String operations are frequent and the temporary string is long (such as greater than or equal to 85000 bytes), a large object heap is allocated, therefore, memory leakage is more likely.

Many people may think about how to optimize the program to reduce the probability of generating a temporary String object. Yes, the appearance of stringbuilder is a logical unit.

 

5. Other experiences

1. Improper use of sessions, especially sessions in inproc mode, should be used to maintain the status. For example, a large user traffic will consume server resources, in addition, the session loss is unstable, so the general site selects restful stateless services;

2. Use more complex data structures, such as nested dictionaries in the dictionary, and dictionary keys and values. I have encountered a very strange project, with at least five layers of dictionary nesting... Some people will argue that the dictionary is a reference type, and automatic garbage collection and so on. In front of Oom, all words are pale;

3. A deep inheritance chain, especially class inheritance, should be clear about GC collection principles if you are familiar with garbage collection, the existence of inheritance may extend the lifecycle of the class, which is not conducive to timely collection. Therefore, if the depth of inheritance exceeds two digits in the actual project, it must be abstract, refactoring is an inevitable choice;

4. Memory leakage is also very common in the development of some multimedia processing programs, such as using GDI + to develop drawing programs, which causes serious memory consumption. In this case, it is extremely important to enable the dispose mode for hosting code;

5. OOM occurs when an index is created during the use of ipve.net. When the data volume goes up, insufficient memory is almost unavoidable, at this time, you must re-adjust the structure to split the index file distribution;

6. Sometimes I call the office component for some report processing and find that the memory seems to be much less at once? If the 7z compression component is used and multiple threads are called, does the memory seem to be tight?

7. Calling third-party mail components to process emails and attachments results in unsatisfactory CPU and memory overhead;

......

You can add more development experiences that may easily lead to oom.

 

6. Be alert to large targets

The situations previously analyzed in this article are based on experience and representation, and there is also a reason for Memory leakage that goes directly to the nature of the problem to be analyzed.

If you have a deep understanding of the memory Collection Principle and the large object and large object heap (LOH), the memory fragmentation problem caused by large objects will be well understood.

To put it simply:

1. Any object larger than or equal to 85000 bytes is automatically regarded as a large object, which is allocated from a special large object heap. The large object heap is terminated and released like the small object heap, but GC is collected.AlgorithmNever compress the memory of a large object heap, because moving 85000 bytes or larger memory blocks in the heap will waste too much CPU time;

2. In. net, CLR uses generation-based garbage collection. large objects are considered part of generation-based generation. GC analyzes which objects are inaccessible, priority analysis is given to 0th and 1st generations. Objects of 2nd generations are generally considered to survive for a long time.

It is precisely because of the two reasons described in 1 and 2 (mainly because of 1st) that memory fragments are easily caused during garbage collection. Here we recommend a widely used article written by foreigners.ArticleFor reference: the dangers of the Loh

As applications run, if LOH causes more and more memory fragments, the effective memory usage will be greatly reduced, for example, in a web application, if the number of temporary objects of strings larger than or equal to 4th bytes is large, the number of users increases as the system runs, GC recovery is under increasing pressure. OOM risks become higher.

Although the OOM caused by memory fragmentation seems unsolvable, if the writer carefully analyzes and solves the problem and tries to reduce the frequency of creating large objects, the memory leakage may be reduced, robust enough programs cannot completely solve Oom, but we can minimize the risk.

A qualified coder must have sufficient preparation and experience in analyzing and solving OOM problems. There are many tools for analyzing and checking Oom, such as ants memory profiler, you can also analyze memory dump files by using a debugging tool such as windbg. Using these tools to make OOM invisible is also a solution.

Finally, this is the last article before the Spring Festival this year. I wish you a happy holiday in advance. The article is so wonderful that you can use the journal account to describe the charge (Charge _ reward) statement. I encourage myself to finish it. Haha.

 

Refer:

<CLR via C #>

The dangers of the Loh