How to locate memory leaks in node. js

Basic knowledge

The memory management of the node. JS process is automatically handled by V8, including memory allocation and deallocation. So when will V8 release the memory?

Within V8, a graph is built for all the variables in the program to represent the association between variables, and when the variable is unreachable from the root node, it means that the variable is no longer being used, or can be recycled.
And this recycling is a process, from the fast GC to the last full GC, it takes a while.
In addition, the full GC has a trigger threshold, so memory may occur for a long time at a high value, or a memory leak, which can be found in the "one node. JS application memory Spike analysis". Another is that the reference does not release, resulting in a failure to enter the GC link, and has always generated new occupancy, which typically occurs at the Javascript level.

Therefore, to locate the memory leak problem, the general solution is to find those that are not used and will not be released variables, processing these variables, the problem is generally solved. If the node. js underlying variable is not released, it can only be resolved by optimizing the startup parameters in addition to submitting issue for resolution.

How to find and solve problem tools

工欲善其事 its prerequisite, we still need some tools to help with the troubleshooting.

Devtool

This is the first node. JS Debugging tool that started this year, and the functions of node. js and Chromium are fused together based on Electron. Operation is more convenient than node-inspector, open Timeline function is more practical, although not real-time display.
You devtool xxx.js can also customize the parameters by using the. DEVTOOLRC only, as described in GitHub

Heapdump + Chrome Devtool

This is a more traditional combination of location-memory leaks. Heapdump can call the Generate memory snapshot directly in the code, then import the snapshot file into Chrome devtool for analysis, and then the operation is almost the same as the former. However, this scheme and the former is a little different is that the former is actually in the browser environment, so the generated memory snapshot has some DOM objects exist, there will be some interference. This scenario is a direct call to the underlying V8 method, and the resulting snapshot is only objects in the node. JS Environment.

Memwatch

This can be used directly in the code, real-time detection of memory dynamics, when a memory leak occurs, will trigger the ' leak ' event, will pass the current heap state, with the Heapdump has the miraculous. See Memwatch.

Process one, reproduce the problem

For garbage collection, the V8 engine has very complex logic to decide when to recycle. Many times, when we find that the memory used by the node. JS process is growing rapidly, it is not possible to determine whether a memory leak is causing it, most likely a programming problem that leads to an unreasonable use of memory. Only when garbage collection triggers, unused memory is freed, and memory growth continues, can we be sure that a memory leak has occurred.

Hidden memory leak problems, most of which are trigger conditions, reproduce the problem is the need for these conditions, so we write code in peacetime, we can be some important aspects of the parameter details printed in the log, so that we reproduce the problem is not to touch the mind, disorderly try.

With parameters that can be used to reproduce the problem, the next step is to identify the problem. We want to determine if this part of the memory is not properly released by the GC. So the question is, how do we know that the program is garbage collected? Obviously, waiting is not the way, we should take the initiative.

In the startup parameters of node. js, a parameter that exposes a manual call to the GC method is provided --expose-gc . After we use this parameter to launch the app, we can invoke the global.gc() garbage collection action manually in code. At the same time, use process.memoryUsage().heapUsed the get memory that the process consumes when it runs. If after the GC, the memory is still not down, you can be sure that the memory leaks.

Second, generate memory snapshot

Since memory is a problem, we need to get a snapshot of the memory running by the program to help locate the problem. But the memory snapshot is not casually hit, there is a certain skill.

We need to generate at least three memory snapshots to better locate the problem. These three times are generated again before the problem occurs, and can be generated two or more times during the duration of the problem.

Why did you do it? It's easy to understand. The first time is to get the stack information under normal circumstances, and after the problem occurs, the stack information will be changed, with the first information, we are good for the later comparison, filtering some useless information. The last two snapshots are used to determine whether the object is problematic than the stack change on an object. The following will be applied in detail.

Third, positioning problems

Use Devtool to ignore the following procedure:

Open Chrome Devtools, go to the Profiles tab, click the Load button, and load the previously generated snapshot.

For memory snapshots, there are four views, summary,comparison,containment,statistics, which are commonly used in the top three.

In the Summary view, we can see all the information for the current snapshot, as well as the information between multiple snapshots. The name of the object's constructor is displayed in the list, ignoring the bracketed object, first observing the other objects, and finally looking at them. The latter shallow size represents the size of the object itself, representing the size of the object retained size and its dependent objects, which are generally not available to the GC.

In the Comparison view, we can make a comparison between multiple snapshots, which is more useful, if we compare the previous two snapshots, it may be faster to locate the problem object. Watch out for new, Deleted, Delta, and if the object is a memory leak, it may have been in new, not Deleted.

In the containment view, we can view the entire GC path, which is of course not used. As you expand each of the items enumerated in Summary and Comparison, you can see the path from the GC roots to this object. Through these paths, you can see what the handle of this object is holding, thus locating the cause of the problem. The note of the value, where the background color is yellow, indicates that the object has a reference in Javascript, so it may not be cleared. If it is red, it means that the object does not have a reference in JavaScript, but it still lives in memory and is generally common in DOM objects, where they are stored in a different location than in JavaScript, and rarely encountered in node. js.

More methods of operation can be seen in this video memory Profiling with Chrome DevTools and memory Management masterclass. There is also Chrome's document Memory Profiling (old) and memory diagnosis (new). The talk is still very detailed. (Please bring your own ladder)

Iv. Problem Solving

Generally in Javascript in the case of a reference to the memory leak, it is better to deal with, just need to be in use after the timely release of the reference.

But the memory problem that exists in the memory spike analysis of node. JS application is a problem of the underlying mechanism, and if you can't wait for bugfix, you can only optimize memory management with some startup parameters first. Common parameters:

• --max-old-space-sizeLimiting the size of the Laosheng area allows you to control the maximum memory footprint, even if a leak occurs, and does not keep memory consumption high. It can be optimized based on the number of open processes and whether the same machine is deployed.
• --gc_globalThis is actually a V8 debug flag, so that GC will always be full GC, the use will have a certain performance loss, depending on the application complexity, different losses.

When we find the problem, after the repair, repeat the above steps to confirm that the problem has been resolved. Sometimes it may not solve the problem at once, so patience is important.

Actual combat

You can download the code used here, GitHub, into the Memory-leak folder.
Let's take an example, apply the above steps to troubleshoot the problem, use the leak-memory example, and the code has another example that you can practice yourself.

Here we use the Devtool for convenience.

devTool leak-memory.js

It then enters the memory snapshot interface in the Open interface to generate the first snapshot. When the console has output, the interval generates two snapshots, resulting in the following.

We switch views, comparing the differences between the next three snapshots, and we can see Foo that the object has been created without being deleted.

We expand Foo , select one of the following instances, look at its GC path, you can see that it has been neverrelease hold the reference (yellow), so it has not been released, then the problem can be processed.

Remove the // neverRelease.splice(index, 1); previous comment, and then repeat the above steps, and you will find that the memory changes are normal.

When using Devtool, you can view the memory timeline of the runtime, and if the image is stepped up, there is usually a problem with the leak. The normal application curve will resemble the sawtooth,

Summarize

1. Memory leak problem positioning, experience is very important, but with the help of good tools, can save a lot of time. If lazy oneself step by step operation, can access Alinode, this can help you very convenient to generate snapshot and other run-time data, and have certain analysis assistance, or convenient.

2. You may see a lot of memory analysis articles that will have some graphs to indicate memory growth, and python can be used to quickly generate related images using matplotlib.pyplot this package.

Reference

• Memory-diagnosis
• Memory Profiling with Chrome DevTools
• Finding a JavaScript Memory Leak in node. js
• A Tour of V8:garbage Collection

reprinted from: http://taobaofed.org/blog/2016/04/15/how-to-find-memory-leak/ Lingheng

How to locate memory leaks in node. js