[Turn] FASTMM use detailed

FASTMM use of the detailed

First, Introduction
FASTMM is applicable to Delphi third-party memory manager, is already famous abroad, in the country there are many people in use or want to use, even Borland also in delphi2007 abandoned their original memory of the accused of the manager, instead of FASTMM.
However, the complexity of memory management and the lack of FASTMM Chinese documents caused many people in the domestic use of many problems, some people gave up the use, I in a recent project used FASTMM, and therefore encountered a lot of problems, after groping and research, finally solved these problems.
second, why to use FASTMM
The first reason is that FASTMM's performance is close to twice times that of the Delphi default memory manager, and you can do a simple test that runs the following code:
Var
I:integer;
tic:cardinal;
s:string;
Begin
TIC: = GetTickCount;
Try
For I: = 0 to 100000 do
Begin
SetLength (S, I + 100);
Edt1. Text: = S;
End
Finally
SetLength (S, 0);
TIC: = gettickcount-tic;
Messagedlg (' tic = ' + IntToStr (TIC), mtinformation, [Mbok], 0);
End
End
On my IBM T23 notebook, using FastMM4 (the latest version of FASTMM) takes about 3300ms, while using the default memory manager, the performance of the 6200MS,FASTMM4 is increased by up to 88%.
The second reason is that the Shared Memory Manager feature of FASTMM is simple and reliable to use. When an application has multiple modules (EXE and DLL) composed of dynamic memory variables such as string transfer between modules is a big problem, by default, each module by its own memory manager, the memory allocated by a memory manager must also be in this memory manager to safely release, Otherwise, a memory error occurs, so that if the memory allocated in one module is released in another module, a memory error occurs. To solve this problem, you need to use the shared Memory manager to use the same memory manager for each module. Delphi's default shared Memory manager is BORLNDMM.DLL, which is unreliable and often problematic, and must be published with this DLL when the program is published. The Shared Memory Manager feature of FASTMM does not require DLL support and is more reliable.
The third reason is that FASTMM also has some accessibility features for helper development, such as memory leak detection, which detects whether the program has memory that is not properly freed, and so on.

Iii. What problems arise
If we develop the application, only one EXE module, then, using FASTMM is a very simple thing, just need to Fastmm.pas (the latest version is Fastmm4.pas) as the first uses unit of the project file, such as:

Program Test;
Uses
FastMM4,
...
However, in general, our applications are made up of an EXE module with multiple DLLs, so that when we pass dynamic memory variables such as string variables across modules, there is a problem, for example, the following test program consists of an EXE and a DLL:

Library test; Test.dll
Uses
FastMM4, ...;
Procedure Getstr (var s:string; const Len:integer); stdcall;
Begin
SetLength (S, Len); Allocating memory
Fillchar (S[1], Len, ' A ');
End
Exports
GETSTR;
-------------------------------------
Program TESTPRJ;
Uses
FastMM4, ...;
//------------------
Unit Mmain; Test interface
...
Procedure Tform1.btndoclick (Sender:tobject);
Var
I:integer;
s:string;
Begin
Try
For I: = 1 to 10000 do
Begin
Getstr (S, I + 1);
Edt1. Text: = S;
Application.processmessages;
End
Finally
SetLength (S, 0);
End
End

A memory error occurs when the Btndoclick process is executed the second time, why? The Delphi string is a reference count, and as with the interface variable, once the reference count is 0, the memory is freed automatically. During the Btndoclick process, the GETSTR procedure is called, and SetLength is allocated a memory for S, at which point the reference count is 1 and then the EDT1 is executed. Text: = s statement, string reference count is 2, loop again calls Getstr to S to reallocate memory, so the reference count of the original string minus 1, and then execute edt1. Text: = S, the original string reference count is 0, then it will be released (note that Is released at TestPrj.exe, not Test.dll release), but there is no error, and when the loop executes, there is a reference count of 2 for the string, but after the execution of SetLength (S, 0), the string is EDT1. The text reference, with a reference count of 1, executes EDT1 the second time the Btndoclick is executed. Text: = S, the last reference count of 1 string reference count minus one to 0 will be released, at this time, a memory error occurs.
Thus, it can be seen that in another module to release the memory allocated by other modules, and does not necessarily occur immediately memory errors, but, if the frequent execution, there will be memory errors, this kind of uncertain error with great concealment, often in the debugging without appearing, but the actual application, not careful analysis is difficult to find the cause.
To solve this problem, we need to find out the root cause is memory management.
Delphi's memory management, Delphi application can use three kinds of memory area: Global memory area, heap, stack, global memory area to store global variables, stack to pass parameters and return values, as well as temporary variables within the function, both of which are automatically managed by the compiler, such as strings, objects, Dynamic arrays are all allocated from the heap, and memory management refers to the management of heap memory, that is, allocating memory from the heap and freeing the memory allocated from the heap (the allocation and deallocation of memory).
We know that a process has only one stack, so it's also easy to mistake a process for just one heap, but in fact, a process can create multiple user heaps, each with its own handle, in addition to a default heap of system allocations (the default size is 1MB). Delphi's memory management is a self-created heap, and Delphi divides a heap into chunks of varying sizes in the form of a linked list, and the actual memory operations are on those blocks.
Delphi defines memory management as allocation (GET), Release (free), and reallocation (Realloc) of memory. The memory manager is also a combination of these three implementations, and Delphi defines the memory manager Tmemorymanager in the system unit:

pMemoryManager = ^tmemorymanager;
Tmemorymanager = Record
Getmem:function (Size:integer): Pointer;
Freemem:function (p:pointer): Integer;
Reallocmem:function (P:pointer; Size:integer): Pointer;
End
As a result, Delphi's memory manager is a Tmemorymanager record object that has three domains that point to the allocation, deallocation, and redistribution routines of memory.
The system unit also defines a variable Memorymanager:
Memorymanager:tmemorymanager = (
Getmem:sysgetmem;
Freemem:sysfreemem;
REALLOCMEM:SYSREALLOCMEM);
This variable is the memory manager of the Delphi program, by default, the memory manager's three domains point to the Sysgetmem, Sysfreemem, Sysreallocmem implemented in Getmem.inc. This memory manager variable is only visible in System.pas, but the system unit provides three routines that can access the variable:

Reads the memory manager, also reads the Memorymanager
Procedure Getmemorymanager (var memmgr:tmemorymanager);
Install the memory manager (that is, replace the default memory manager with the new memory manager)
Procedure Setmemorymanager (const memmgr:tmemorymanager);
Whether the memory manager is already installed (that is, if the default memory manager has been replaced)
function Ismemorymanagerset:boolean;
four, shared memory manager
What is a shared memory manager?
The so-called shared Memory Manager, is an application of all modules, whether EXE or DLL, use the same memory manager to manage memory, so that the allocation and release of memory is the same memory manager to complete, there will be no memory error problems.
So how do you share the memory manager?
From the above analysis, we can know that since the use of the same memory manager, then simply create a separate Memory Manager module (DLL), all other modules use the module's memory manager to allocate and free memory. Delphi7 the default is to take this approach, when we use the wizard to create a DLL project, the project file will have this passage:
{Important Note about DLL memory Management:sharemem must is the
First unit in your library ' s USES clause and your project ' s (select
Project-view Source) USES clause If your DLL exports any procedures or
Functions that pass strings as parameters or function results. This
Applies to all strings passed to and from your dll--even those
is nested in records and classes. SHAREMEM is the interface unit to
The BORLNDMM. DLL shared Memory manager, which must be deployed along
With your DLL. To avoid using BORLNDMM. DLL, pass string information
Using PChar or shortstring parameters. }
This passage prompts us, Sharemem is BORLNDMM. DLL Shared Memory Manager Interface unit, let's take a look at this sharemem, this unit file is very brief, which has such a declaration:

Const
Delphimm = ' Borlndmm.dll ';
function Sysgetmem (Size:integer): Pointer;
External Delphimm name ' @[email protected] $qqri ';
function Sysfreemem (p:pointer): Integer;
External Delphimm name ' @[email protected] $QQRPV ';
function Sysreallocmem (p:pointer; Size:integer): Pointer;
External Delphimm name ' @[email protected] $QQRPVI ';
These statements guarantee that the BORLNDMM.DLL will be statically loaded.
The initialization in Sharemem is such a code:
If not Ismemorymanagerset then
Initmemorymanager;
First determine if the memory manager has been installed (that is, if the default memory manager is replaced), if not, initialize the memory manager, Initmemorymanager is also very simple (remove the useless code):

Procedure Initmemorymanager;
Var
Sharedmemorymanager:tmemorymanager;
Mm:integer;
Begin
Force a static reference to Borlndmm.dll, so we don ' t has to LoadLibrary
Sharedmemorymanager.getmem:= Sysgetmem;
MM: = GetModuleHandle (DELPHIMM);
sharedmemorymanager.getmem:= GetProcAddress (MM, ' @[email protected] $qqri ');
sharedmemorymanager.freemem:= GetProcAddress (MM, ' @[email protected] $QQRPV ');
sharedmemorymanager.reallocmem:= GetProcAddress (MM, ' @[email protected] $qqrpvi ');
Setmemorymanager (Sharedmemorymanager);
End
This function defines a memory manager object, sets the three function implementations of the domain to Borlndmm.dll, and then calls Setmemorymanager to replace the default memory manager.
In this way, regardless of the module, because the SHAREMEM as the first uses unit of the project, therefore, each module of SHAREMEM initialization is the first executed, that is, each module memory manager object, although not the same, but, The memory manager's three function pointers are all function implementations that point to Borlndmm.dll, so the memory allocation and deallocation of all modules is done inside Borlndmm.dll, so there is no problem of releasing memory across modules causing errors.
So how does FASTMM implement a shared memory manager?
FASTMM took a very simple principle, is to create a memory manager, and then put the address of the memory manager in a process where all the modules can read the location, so that the other modules before the creation of memory manager, first check whether there are other modules have put the memory manager in this location, If you are using this memory manager, create a new memory manager and place the address in this location so that all modules of the process use a memory manager to share the memory manager.
Moreover, this memory manager is not sure which module is created, all modules, as long as the FASTMM as the first uses unit of its engineering files, it is possible that the creator of this memory manager, the key is to see its loading order in the application, the first loaded module will become the creator of the memory manager.
So, how is FASTMM specifically implemented?
Open Fastmm4.pas (the latest version of FASTMM), or see its initialization section:

{Initialize All the lookup tables, etc. for the Memory manager}
Initializememorymanager;
{Have another mm been set, or has the Borland mm been used? If So, the This file
Is isn't the first unit in the uses clause of the project ' s. DPR file.}
If Checkcaninstallmemorymanager Then
Begin
Installmemorymanager;
End
Initializememorymanager is the initialization of some variables, which is done after the call Checkcaninstallmemorymanager detects if FASTMM is the first uses unit of the project, and if True is returned, Then call Installmemorymanager to install FASTMM's own memory manager, and we'll extract the key code for that function in order for analysis:
{Build A string identifying the current process}
Lcurrentprocessid: = GetCurrentProcessId;
For I: = 0 to 7 do
uniqueprocessidstring [8-i]:= hextable [(Lcurrentprocessid shr (i * 4) and $F)];
Mmwindow: = FindWindow (' STATIC ', PChar (@UniqueProcessIDString [1]));
First, get the ID of the process, convert it to a hexadecimal string, and then look for the window with that string as the window name.
If the window is not yet in the process, Mmwindow returns 0, and the window is created:
Mmwindow: = CreateWindow (' STATIC ', PChar (@UniqueProcessIDString [1]),
Ws_popup, 0, 0, 0, 0, 0, 0, hinstance, nil);
What's the use of creating this window, keep looking at the following code:

If Mmwindow <> 0 Then
SetWindowLong (Mmwindow, Gwl_userdata, Integer (@NewMemoryManager));
Newmemorymanager.getmem: = Fastgetmem;
Newmemorymanager.freemem: = Fastfreemem;
Newmemorymanager.reallocmem: = Fastreallocmem;
As you can see from MSDN, each window has a 32-bit value that can be used by the application that created it, either by calling SetWindowLong with the Gwl_userdata parameter or by using the Gwl_ UserData is called GetWindowLong for parameters to read. As a result, it is very clear that FASTMM saved the address of the memory manager to be shared to this value, so that other modules can be read to this value through GetWindowLong, thereby obtaining a shared memory manager:

Newmemorymanager: = pMemoryManager (GetWindowLong (Mmwindow, gwl_userdata)) ^;
From the above analysis, it can be seen that fastmm in the implementation of the shared memory manager is much more ingenious than Borland, Borland implementation method so that the application must publish BORLNDMM.DLL together, and FASTMM implementation method does not require any DLL support.
However, the above excerpt code ignores some compile options, and in fact, to use FASTMM's shared Memory Manager feature, you need to open some compile options on the Fastmm4.pas unit when each module compiles:
{$define Sharemm}//is to open the Shared Memory Manager feature and is the prerequisite for the other two compilation options to take effect
{$define Sharemmiflibrary}//is to allow a DLL to share its memory manager, if this option is not defined, only EXE modules in an application can create and share their memory manager, because statically loaded DLLs are always loaded earlier than EXE, so , you must turn on this option if a DLL is statically loaded, or you may get an error
{$define ATTEMPTTOUSESHAREDMM}//is a memory manager that allows one module to be shared with another module
These compilation options are defined and explained in the Fastmm4options.inc file in the directory where Fastmm4.pas is located, except that the definitions are commented out, so you can uncomment to open these compilation options, or you can create an. inc file in your project directory (such as Fastsharemm . Inc), write these compilation options to this file, and then add "{$include fastsharemm.inc}" before the "{$Include fastmm4options.inc}" at the beginning of Fastmm4.pas, so that Different projects can use different fastsharemm.inc files.
five, multi-thread memory management
is memory management secure in a multithreaded environment? Obviously, if certain measures are not taken, it is certainly not safe, and Borland has taken this into account, so A system variable Ismultithread is defined in Delphi's System.pas, which indicates whether it is currently a multithreaded environment, so how does it work? The code that opens the Tthread.create function can see that it called Beginthread to create a thread, and Beginthread sets the Ismultithread to true.
Then look at the implementation of Getmem.inc's Sysgetmem, Sysfreemem, and Sysreallocmem, which can be seen at the beginning by such statements:
If Ismultithread then EnterCriticalSection (Heaplock);
In other words, in multi-threaded environment, the allocation and release of memory must be synchronized with the critical section to ensure security.
Instead, FASTMM uses a cup command lock to synchronize, which, as a prefix to other instructions, can lock the bus during the execution of a single instruction and, of course, synchronize if Ismultithread is true.
and ismultithread is defined in the SYSTEM.PAS system variables, each module (EXE or DLL) has its own ismultithread variable, and, by default, Fasle, only the user thread created in the module, this variable is set to True, so we create in EXE Building a thread will only set the Ismultithread in the EXE to TRUE, and will not set ismultithread in the other DLL modules to true, but, as mentioned earlier, if we use a statically loaded DLL, These DLLs will be loaded by the system earlier than EXE, when the first loaded DLL will create a memory manager and share it, the other modules will use this memory manager, that is, EXE's ismultithread variable does not affect the application's memory manager, The memory manager also thinks that it is not currently a multithreaded environment, so there is no synchronization, so there is a memory error.
The solution to this problem is to find a way. When in a multithreaded environment, all modules are ismultithread to true to ensure that regardless of which module actually creates a memory manager, the memory manager knows that the current multithreaded environment requires synchronous processing.
Fortunately, Windows provides a mechanism that allows our DLLs to know that the application created the thread. DllMain function is the entry function of DLL dynamic link library, Delphi encapsulates this entry function, provides a Tdllproc function type and a variable dllproc of that type:

Tdllproc = procedure (Reason:integer); Defined in System.pas
Defined in Sysinit.pas:
Var
Dllproc:tdllproc;

When the system calls the DLL's DllMain, Delphi will finally call Dllproc for processing, dllproc can be pointed to our own TDLLPROC implementation. And when the process creates a new thread, the operating system calls DllMain with the Reason=dll_thread_attach parameter, and then Delphi calls Dllproc with that argument. So we just implement a new TDLLPROC implementation thisdllproc and point Dllproc to Thisdllproc, while in Thisdllproc, when you receive Dll_thread_ Set Ismultithread to True when attach. The implementation code is as follows:

library xxx;
Var
Olddllproc:tdllproc;
Procedure Thisdllproc (Reason:integer);
Begin
If Reason = Dll_thread_attach Then
Ismultithread: = True;
If Assigned (Olddllproc) Then
Olddllproc (Reason);
End
Begin
Olddllproc: = Dllproc;
Dllproc: = Thisdllproc;
Thisdllproc (Dll_process_attach);
Vi. Summary
This article mainly discusses the following issues:
1, why to use FASTMM
2. What happens when you pass dynamic memory variables across modules?
3, Delphi memory management and memory manager is going on
4, why to share the memory manager, Delphi and FASTMM, respectively, how to implement the memory manager shared
5, multi-threaded environment, memory manager how to achieve synchronization
6, multi-threaded environment, how to set the Ismultithread variable across modules to ensure that the memory manager will synchronize

To use FASTMM correctly, you need to do the following when the module is developed:
1. Open Compilation Options {$define Sharemm}, {$define sharemmiflibrary}, {$define attempttousesharedmm}
2, the FASTMM (4). Pas as the first uses unit for each project file
3, if it is a DLL, need to handle the DllMain call with Dll_thread_attach as parameter, set Ismultithread to True

Vii. Reference Documents
"Windows Programming Fifth Edition" []charles Petzold, Peking University Press
"Delphi Source Code Analysis" greatlypraised, electronic industry Press

[Turn] FASTMM use detailed