How to break through the address space limit in 32-bit programs to use memory larger than 4 GB

Http://bbs.51cto.com/thread-464670-1.html

 

How Program All 32-bit applications have 4 GB of process address space, because 32-bit addresses can map up to 4 GB of memory (for friends who are not familiar with the concept of virtual address space, I suggest you go to Windows core programming ). For Microsoft Windows operating systems, applications can access 2 GB of process address space (32-bit Linux can access 3 GB address space), which is called a virtual address space in user mode. In this 2 GB user mode, the virtual address space is located at half of the 4 GB address space, and the corresponding one and a half 2 GB address space is used by the operating system kernel, therefore, it becomes a virtual address space in kernel mode. In a process, all threads read and share the same 2 GB user mode virtual address space.
For general applications, 2 GB of address space is enough, but for some special applications that require massive memory usage (a typical example is a database system, 2 GB address space is far from enough. To alleviate the shortage of address space, Microsoft provides an appropriate solution. All operating system versions starting with Windows 2000 server provide a boot. the INI startup switch (/3 GB) can provide applications with the capability to access 3 GB of process address space, thus limiting the address space in kernel mode to 1 GB. The following is an example of the boot. ini file with the 3 GB option enabled: # _ "['N'-R:} 5 K (C; {; P
7 A (y # N4 P: T (P (t
[Boot loader]
Timeout = 30 A (s )\! U %]; A/L
Default = multi (0) disk (0) RDISK (0) Partition (1) Windows 1 A G8}; q) g %} $ L
[Operating systems]
Multi (0) disk (0) RDISK (0) Partition (1) Windows = "Windows Server 2003, enterprise"/fastdetect/3 GB
Although the/3 GB option can increase the address space in user mode by 50% (from 2 GB to 3 GB), for applications such as database systems, the increase of 1 GB of address space can only be a cup of water, but it cannot solve many problems, and because the operating system kernel can only use 1 GB of address space, this may have a negative impact on the operation of the operating system. Therefore, unless there is no better solution, the/3 GB method is not recommended.
/L # K (S % W "n
* U9 t 'G) g (y0 J0 O9 _ 1 J0 Q! C
Given the need for massive memory for applications such as database systems, Intel also feels that 4 GB of memory is insufficient, therefore, the memory address line in the CPU chip is extended from 32 to 36 (that is, up to 64 GB), which is called the physical address extension (PAE: physical address extension ). PAE allows the operating system or application to use a maximum of 64 GB physical memory. For Windows systems (more than 2000. use the/PAE option in the INI file (similar to the/3 GB option above ). We need to remind you that, if not in Boot. if the/PAE option is used in the INI file, even if the computer has been configured with more than 4 GB of physical memory, it cannot use more than 4 GB of memory in the Windows operating system (in fact, according to my experience, if the/PAE option is not selected, Windows can only identify up to GB of physical memory. Why not 4 GB? If you know, please let me know ).
Although PAE makes it possible to use more than 4 GB physical memory in applications, the virtual address space of 32-bit applications does not change with the increase of physical memory, this means that you cannot use virtualalloc (getcurrentprocess, 2 GB ,...,...) such a function = directly allocates a memory area that is close to the user mode address space. To break through the 32-bit address space limitation, an awe: address processing wing extensions mechanism is required (see ).
Awe is a group of extensions of Windows Memory Management functions. It enables applications to use more than 2-bit memory that can be used through standard 32-bit addressing ~ 3 GB memory. Awe allows applications to obtain physical memory and dynamically map non-Paging memory views to 32-bit address space. Although the 32-bit address space is limited to 4 GB, the non-Paging memory can be much larger than 4 GB. This allows applications that require a large amount of memory (such as large database systems) to use much larger memory than the 32-bit address space.
When using the awe mechanism, pay attention to the following points:
(1) awe allows a 32-bit architecture to allocate more than 4 GB of physical memory. Awe should be used only when the physical memory available to the system is larger than the virtual address space in user mode. 1 G6 l-I; A3 z0 B7} $ S2 m
(2) to enable the 32-bit operating system to support more than 4 GB physical memory, you must enable the/PAE option in the boot. ini file.
(3) If. if the/3 GB option is enabled in the INI file, the operating system can use up to 16 GB physical memory. Therefore, if the actual physical memory exceeds 16 GB, make sure that the/3 GB option is not used.
(4) The Memory allocated by awe is non-Paging physical memory, which means that this part of memory can only be exclusively used by the allocated application and cannot be used by the operating system or other programs, until these memories are released, this is significantly different from the virtual memory allocated by the common virtualalloc function, which is not involved in paging replacement. 5] # f) C, _ 5 o $ H8 Z
In Windows, awe-related API functions include the following: -K $ H1 R0 \ & P, u

bool allocateuserphysicalpages (
handle hprocess,
pulong_ptr numberofpages,
pulong_ptr userpfnarray # r$ H, M &] 0 i8 l % d! D
);
$ L4 P %} 4 J * A; D! B % Z & H) B
bool winapi allocateuserphysicalpagesnuma ( 9 I '| & @) t' I. U
handle hprocess,
pulong_ptr numberofpages, + I5 o 'K, S *?) _ 0 d-S
pulong_ptr pagearray,
DWORD nndpreferred $ X! @-'5 K' R) g, X "G
);

Bool mapuserphysicalpages (8 W; '# N/' 8 n8 T # P # d
Pvoid lpaddress,
Ulong_ptr numberofpages,, D9 o "V + D! O/q) R1 Q7 y
Pulong_ptr userpfnarray"~ +? 8 _ "L $ F6 R) h j. E6 ~
);; S 'e-F: H, | # J
* R2 F6 X2 B7 d) \ 9 W
Bool mapuserphysicalpagesscatter ((O + V ;? 7 K7 o "A9 N/W
Pvoid * virtualaddresses,) '$ C/C' R3 s! I (A. |
Ulong_ptr numberofpages,
Pulong_ptr pagearray
);

Bool freeuserphysicalpages (
Handle hprocess,
Pulong_ptr numberofpages,
Pulong_ptr userpfnarray
);
$ O * P #_! G0 C, R & T8 V2 s
For more information about the parameter meanings of each function, see msdn. allocateuserphysicalpagesnuma is a newly added function for Windows Vista and Windows 2008 server. It supports NUMA (non-consistent memory access ). The following describes how to use these API functions to achieve memory exceeding 4 GB. ! J) S/I % s "|) l-u/[3 A; | * I
Use the allocateuserphysicalpages function to allocate the required physical memory as follows: 6 y (Z9 D6 L4 {$ {
# F3 g'l + u "L (B & N/T
Ulong_ptr numberofpages = xxx; // Number of memory pages to be allocated% J % _ (E c 'c7 Q ,~
Ulong_ptr * apfns = new ulong_ptr [numberofpages];
Bool bresult = allocateuserphysicalpages (getcurrentprocess (), & numberofpages, apfns ); , C k + S: V1 S6 E1 m
* U. _ # D. Z1 y, p, q) M: R
// Check whether the allocated memory is successful * C8 N) T + P.} (Y1 F % O3 A7 s
If (! Bresult)
{4 N6 Z "? $ Y4 f * z-S1 s
// Identify allocation and handle errors
//.....
}

// Check the number of actually allocated memory pages
If (numberofpages! = Xxx) /X B7 y0 K5 ~ 5 [+ P: K + I-P
{ % H! K) K4] m
//....
}
Note that Code The user must have the "Lock Pages In Memory" permission. This permission allows you to use a process to keep data in the physical memory. This prevents the system from paging data to the virtual memory on the disk. Exercise this permission will significantly affect system performance by reducing the number of available random access memory (RAM. You need to grant this permission to the user in the Local Security Policy Management Program, as shown in:
! F) S5 W' ^) W: t' G0 S & X
After assigning the preceding permissions to the user, you must use the code to enable the permissions in the program, as shown below: 7 K. F, P! M5 W $ H: W
$]: T % Z: N "N) K6 J $ D6 K7 E: Y
// Set the permission to lock the physical memory. This code is executed before allocateuserphysicalpages is called. X) | 2 ~, O! A + Z + I
If (! Awesetlockpagesprivilege (getcurrentprocess (), true )) 3 | 3 L6 L + Y. Q & N2 U/\ (B + k
{ 1 C & O) I (B0 W & y6 L: k's
// Output error message:~ /U: d1 V (D5 J1 S0
..........
} $ J2 K (V * '6 N, T % x! A

///
/// Set or clear the Memory Lock permissions required to enable awe (address drawing wing extensions. 8 N6 K! J m (S "E4 P
///
/// 1 Q ^ 0 Y2 Q, Z! W :\
/// Process handle.
/// 2 Q + X8 F3 N/W3 g
///9 t; N5 L (Y1 m) I3 J
/// Set or clear the flag.
/// U (S-G0 s! } 'V2 d
/// (J, ^. s # M, y9 T4 h $ '! E9 Y8 P4 |
/// If successful, true is returned; otherwise, failure is returned.
/// , E p-@ 7 l! K/T4 J6 H/^ 4 J
Bool awesetlockpagesprivilege (handle hprocess, bool enable)
{
Handle token = NULL;2 K & N $ J "[$ x) |: F
Bool result = false; & I/K9 V5 N) I6] 6 D * y9 C (K
Token_privileges info = {0 }; : T (Q (\ "L & T. A5 e l: M5 G5 V # C
4 M5 X-D/X' J # E
// Open the token ; Q9 y! H7 {(K! E
Result = openprocesstoken (hprocess, token_adjust_privileges, & token );& ^. O9 P8 H + ^ 4 O4 E
If (! Result)
Return false; 6 Y3 Q # H + n t o & M; n! N
(''J-W. v # R
// Set the permission information : G + I2 B-R1 @. p * x/I U % u % {
Info. privilegecount = 1;
Info. Privileges [0]. Attributes = Enable? Se_privilege_enabled: 0; $? 3] p-e 'J/U2G & K $ v

// Obtain the ID of the locked memory permission
Result = lookupprivilegevalue (null, se_lock_memory_name, & (info. Privileges [0]. luid )); 3 R7 S /? : A * j; T1 | + W; h.p A; j
If (! Result) K-T1 n % I3 G $ ', J $ G (Q & w9'
{
Closehandle (token );
Return false;
}
, S, I-\. Z # Y5 w6 A #
// Adjust permissions 6 A %} 0 C' o! L
Result = adjusttokenprivileges (token, false, (ptoken_privileges) & info, 0, null, null );
If ((! Result) | (getlasterror ()! = Error_success ))
{ 9 P6 Y5 T (Q-P & S & ^ + c
Closehandle (token ); -_! B; L + {$]
Return false;
} ! C v (] 1 K. H & F: G8 H0 U $ s; q
1 @ 'j '? (] 5 C' K8 U6 O5 ^
// Return successfully% @! W; p l s % F "O5 I
Closehandle (token );
Return true;
} (W "l 'n8 R8 R5 [# w6 h.j

After the physical memory is allocated using allocateuserphysicalpages, the next step is to map the physical memory into the user mode address space using the mapuserphysicalpages or mapuserphysicalpagesscatter function. The usage of these two functions is similar, but the first parameter is different. Because the physical memory size of the sub-configuration exceeds the size of the user mode address space, it is obviously impossible to map all the physical memory to the address space at a time. The common practice is to allocate a small contiguous area (address window) in the user mode address space, and then dynamically map some physical memory to the address space according to the needs, this is the true meaning of the word "address window extension. The sample code is as follows:6 d, D6 M4 C; u7 H8 H

// Defines the 16 M address window'D; r T8 C "F3 Q % O & O, U + q (J; m
# Define memory_requested (16*1024*1024)
; O % [! M # F % Q
// Address Allocation window
Pvoid lpmemreserved = virtualalloc (null, memory_requested, mem_reserve | mem_physical, page_readwrite );# K, d '? # V8 D +])] 'y

// Map the physical memory to the address space (the page mapped each time varies according to your needs,
// The third parameter apfns of the following function points to different physical pages)
Bresult = mapuserphysicalpages (lpmemreserved, numberofpages, apfns );5} 8 I) R'}/I (T % T. E8 D & I

// The following uses the lpmemreserved pointer to operate the physical memory just like the normal memory.
...................
After use, you can use freeuserphysicalpages to release the allocated physical memory, for example:

// Cancel memory ing
Bresult = mapuserphysicalpages (lpmemreserved, numberofpages, null );
; L # '% K & R-H-\/K &
// Release the physical memory
Bresult = freeuserphysicalpages (getcurrentprocess (), & numberofpages, apfns );8 H8 Z. I; N7 T # F-H3 G5 S3 {! H) I + Y

// Release address window
Bresult = virtualfree (lpmemreserved, 0, mem_release );'\ & | "X K"] 9 m3 C $}

// Release the physical page number Array
Delete [] apfns;
One of the scenarios where the awe mechanism is most used is the buffer manager of the database system, for example, the memory manager of SQL Server. Although the above Code is based on the Windows operating system, the PAE and awe mechanisms are not unique to Windows, and 32-bit Linux also has similar APIs. For examples of the complete awe mechanism, you can refer to the MySQL source code.
Finally, the good news for developers is that 64-bit CPUs and operating systems are becoming increasingly popular. In a 64-bit environment, the address space of a process's user mode can be up to 8 Tb (that is, many 64-bit systems currently only use 40-bit memory addresses, far from making full use of 64-bit memory addresses), we will not worry about insufficient address space for a long time in the foreseeable future, let's cheer for the arrival of the 64-bit era!