Troubleshooting and solving the problem of platform server handle leaking

We monitor the platform has a alarm server, its main function is to receive the front end, TDDC, network Management servers, such as the alarm sent, and according to the alarm linkage configuration for the corresponding linkage operation, recently found in the server running process, through the Task Manager to view the number of its handle will continue to increase, so that the impact of other server work, Initial suspicion is a handle leakage problem, it is now analyzed and checked.

A handle is a unique integer that Windows uses to identify objects that an application establishes or uses, and the kernel objects of Windows include threads, windows, bitmaps, GDI objects, and so on. The application accesses the kernel object through a handle, and when the kernel object is exhausted, it needs to release the resource to close the kernel object handle, which can cause a handle leak if it fails to close properly.

In general, if a handle leak is suspected, the most important task is to look for a leaked handle type, which helps to troubleshoot the analysis later and narrow the target range. There are some tools to help with the analysis, such as process explorer,pchunter, which can be very clear about the kernel objects that the process is using and can help us find the problem.

Open process Explorer and locate the running Alert server process, which displays the kernel object handle currently created by the alarm server, and you can see that the number of handles for the process is already thousands and is growing rapidly.

The handy thing about this tool is that when a new kernel object is created, it is identified in green in the list box below, it is easy to see, observe a period of time to discover, the constant thread object is created, and does not close, the preliminary guess should be that the handle of the threading object is not closed. The next task is to find out where this thread kernel object was created.

Finding the location where the handle was created can be obtained by WinDbg, WinDbg is the next very powerful kernel debugger in Windows. Use the WinDbg!htrace command to debug a handle disclosure. The principle is relatively simple, that is, the process of the kernel object to do two snapshots, compare the two different, you can know which kernel objects are created, but also can find where to allocate.

Open WinDbg, press F6, attach the alarm server process. Enter!htrace–enable on the command line to turn on the Htrace function. Enter!htrace–snapshot to do the first snapshot, and then enter the G command to allow the program to execute for a period of time.

The process is as follows:

After the program runs for a period of time, press the Ctrl+break key to break the program down. When you enter!handle–diff, you can compare the allocation context of the new handle.

By careful observation, you can see that there are multiple thread kernel objects created, and the!handle command can be used to view information about a kernel object. For example, to view a thread kernel object with a handle value of f98, you can enter!handlef98 ff. As shown

Its starting address is hpr_getnetworkflowdata, and you can determine that the thread was created using the HPR library, and you can narrow the scope by looking for hpr_thread_create in the source code.

You can also see that the thread ID is 7670.FEF4, where 7670 is the thread ID of the process ID,FEF4, so you can switch to that thread to see, enter

The ~~[fef4]s command toggles the thread.

There is an error message that the thread is no longer there, so we can basically make sure that there is a leak, the thread is no longer there, but the thread's kernel object failed to close.

By careful observation, we can see that these thread objects are created by the same thread,

It eventually points to the 71a8 thread, so we can be sure that 71a8 this thread creates a lot of thread kernel objects, but does not close the kernel object's handle, which causes the thread kernel object's handle to be leaked, so long as it locates to 71a8 this thread, it can find the leaking place.

Enter ~~[71a8]s, switch to this thread, and enter KB to print its call stack, such as:

The call stack is a bunch of very strange data, in fact, because my system is 64-bit, and the alarm server this process is 32-bit, WinDbg use 64-bit context to resolve the 32-bit process, resulting in error parsing. You can then switch to the 32-bit context by!SW this command. And then typing the KB command, you can see the call stack for that thread,

Can clearly see the call stack of the thread, at this time the thread is waiting for the socket connection, using the HPR Library's hpr_accept function, to this point, it is very easy to locate the thread function in the source code.

Search the source code for the Hpr_accept function, with only one place:

It is clear that after the thread accepts the connection, it creates a thread, but does not close the handle of the created thread, resulting in a handle leak, which is consistent with our previous analysis. Knowing the problem, the amendment is very convenient.

Citation: http://blog.csdn.net/shuizhilan/article/details/46632083

Troubleshooting and solving the problem of platform server handle leaking